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ABSTRACT 



EVALUATION OF THE CORRELATED SCIENCE AND MATHEMATICS 
PROFESSIONAL DEVELOPMENT MODEL, 2009-2010 COHORT 

Rebecca Morlier, M.S. 

The University of Houston-Clear Lake, 2012 

Thesis Chair: Sandra Browning, Ph.D. 

The purpose of this paper is to evaluate the effectiveness of the 2009-2010 iteration of the 
Correlated Science and Mathematics (CSM) professional development program which 
provides teachers and principals experience with integrated and effective science and 
mathematics teaching strategies and content. Archival CSM data was analyzed via mixed 
methods. Following the five-level evaluation model of Guskey (2000), the 2009-2010 
CSM program was effective. Teacher participants enjoyed CSM and gained content 
knowledge and pedagogical skills; teachers demonstrated increased use of effective 
teaching practices championed by CSM, and as measured by pretests and posttests, 
demonstrated significant gains in physics and mathematics. Although principals 
participated, the strong efforts to garner organizational support were only minimally 
effective. As measured by pretests and posttests, students of participant teachers 
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demonstrated significantly greater achievement in mathematics than students of control 
teachers; no correlation can be made regarding student achievement in science. 
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Chapter I 
Introduction 

National and state standards call for integration of science and mathematics 
disciplines in the school curriculum (American Association for the Advancement of 
Science [AAAS], 1989, 1993; Berlin & White, 1992; National Council of Teachers of 
Mathematics [NCTM] 1989, 2000; National Research Council [NRC] 1996; Texas 
Education Agency [TEA], 2010). Integration was incorporated into the national 
standards as early as 1989, yet integration teaching practices are still rarely implemented 
in the classroom (Merrill, 2001). Theoretical support for integration is provided by a vast 
body of research over the past century but limited empirical research on the effects of 
integration on student outcomes exists (Berlin & Lee, 2005; Hurley, 2001). Teachers of 
science and mathematics often lack content knowledge, content pedagogy, and 
confidence in mathematics or science, respectively, which hinders their use of integrated 
lessons and teaching strategies (Berlin, 1994; Frykhohn & Glasson, 2005; Fumer & 
Kumar, 2007; Stinson, Harkness, Meyer, & Stallworth, 2009). Teachers also have little 
experience learning via integrated curricula themselves and have no model from which to 
base and compare their attempted integrated lessons on or with (Frykhohn & Glasson, 
2005; Huntley, 1998, 1999; Lehman, 1994; Lehman & McDonald, 1988; McBride & 
Silverman, 1991; Pang & Good, 2000; Stinson et al., 2009). Few professional 
development models have been proposed or successfully implemented that address these 
needs and help teachers develop integration teaching skills (Yasar et al., 2006). 

The Correlated Science and Mathematics (CSM) professional development model 
provides teachers with integration experience and teaching strategies for correlating or 
integrating science and mathematics in their teaching practices (West & Browning, 

2010). CSM is standards-based and incorporates inquiry practices. CSM is unique in 
that it emphasizes the importance of the proper use of language in science and 
mathematics teaching and learning. Science and mathematics content and processes 
overlap and support each other in many ways (AAAS, 1989; Berlin & White, 1992; 
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McBride & Silverman, 1991; NCTM, 2000), but also use similar language in disparate 
ways which confuses students (Austin, Converse, Sass & Tomlins, 1992; Oyoo, 2009). 
Principals are also included in CSM professional development; CSM teaches 
administrators the characteristics of effective integrated and inquiry lessons and offers 
strategies for coaching and supporting teachers in their use of integrated and inquiry 
practices. 

The purpose of this thesis is to evaluate the effectiveness of the 2009-2010 CSM 
professional development program during its fourth year of field testing. Effectiveness 
will be assessed by following Guskey’s (2000) five-level evaluation model to detennine 
how well the CSM program goals were met, how well the 2009-2010 CSM professional 
development met accepted standards for effective professional development, and how 
participant teachers and principals and their students were affected. 

Research Questions 

The evaluation of the 2009-2010 CSM professional development program will 
address how the program has affected participant teachers, their students, and participant 
principals. 

• Effect on Teachers 

1) What were participant teachers’ perceptions of the CSM program and 
strategies? 

2) Does participant teacher content knowledge of physics and mathematics, 
as measured by pretests and posttests, significantly improve as a result of 
the CSM summer institute? 

3) To what extent do participant teachers incorporate CSM strategies into 
their classroom practices? 

• Effect on Principals 

4) What were participant principal perceptions of the CSM program and 
strategies? 

5) How did participation in CSM affect principals’ management of science 
and mathematics teachers? 

• Effect on Students 
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6) How does the performance of students of participant teachers, as measured 
by the percentage of students who met Texas standards on the science and 
mathematics Texas Assessment of Knowledge and Skill (TAKS) 
examinations, compare to (a) the performance of students of non- 
participant teachers, and (b) the performance of students of participant 
teachers prior to CSM training? In other words, did CSM indirectly 
influence student achievement on TAKS? 

7) Is there a significant difference in the degree of achievement between 
students of participant teachers and students of non-participant teachers as 
measured by science and mathematics pretest and posttest scores? 

This research will serve as both a summative assessment of 2009-2010 
implementation of the CSM professional development program and a formative 
assessment of the entire program to date. Evaluation results will identify strengths and 
weaknesses of CSM professional development design and implementation. Suggestions 
for program improvement will allow CSM directors to modify the program for future 
cycles. Demonstration of continued success in affecting teacher change and a correlation 
with improved student achievement in science and mathematics will warrant 
implementing the CSM professional development program on a larger scale such that a 
greater number of teachers and students may benefit. At a minimum, effective portions 
of the CSM model will be clarified such that other professional development programs 
can incorporate CSM best practices into other contexts. 

Definition of Terms 

Professional development: The NSDC (2010b) defines professional development 
as “a comprehensive, sustained, and intensive approach to improving teachers’ and 
principals’ effectiveness in raising student achievement” (par. 3). Guskey (2002b) states 
that professional development incorporates “systematic efforts to bring about change in 
the classroom practices of teachers, in their attitudes and beliefs, and in the learning 
outcomes of the student” (p. 381). 

Integrated: For the purposes of this research, integrated will refer to lessons that 
combine science and mathematics such that the content of one discipline is supported by 
the content or use of the other discipline (Berlin & White, 1992; McBride & Silvennan, 
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1991). The degree of science and mathematics melding may fall anywhere along the 
Science and Mathematics Continuum (see Figure 1) (Huntley, 1998; Lonning & 
DeFranco, 1994; Lonning, DeFranco, & Weinland, 1998). Maximum integration would 
fall at the center of the Continuum where the content of two disciplines would be taught 
in total concert, in equal proportion. 




Figure 1. Mathematics and Science Continuum. Each box summarizes the content focus 
of integration at points along the Continuum. Adapted from Huntley (1998) and Lonning 
and DeFranco (1994). 

Correlated: Correlated lessons allow teachers to facilitate the learning of science 
and mathematics together such that a bystander may not readily discern if the class was 
labeled science or mathematics (Vasquez-Mireles & West, 2007). Correlated lessons do 
fall in the maximum integration range of the Mathematics and Science Continuum 
because mathematics and science content are taught synergistically, and as such, 
correlation is a type of integration. To be categorized as correlated, an integrated lesson 
must additionally incorporate seven specific characteristics. These facets include 
teaching for conceptual understanding, consistently using each discipline’s proper 
language, linking the disciplines naturally, clarifying the disciplines’ parallel ideas, 
clarifying language students find confusing, basing objectives on standards, and 
incorporating 5E inquiry strategies when appropriate (West & Browning, 2010). 
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Chapter II 

Review of the Literature 

The integration of science and mathematics instruction in classrooms is called for 
by national and state education standards (AAAS, 1989, 1993; Berlin & White, 1992; 
NCTM 1989, 2000; NRC 1996; TEA 2010). Although integration has been promoted for 
years, it remains underutilized as a teaching strategy (Merrill, 2001) because barriers 
exist which hinder the implementation of integration teaching strategies (Berlin, 1994; 
Frykhohn & Glasson, 2005; Fumer & Kumar, 2007; Huntley, 1998, 1999; Lehman, 

1994; Lehman & McDonald, 1988; McBride & Silverman, 1991; Pang & Good, 2000; 
Stinson, Harkness, Meyer, & Stallworth, 2009). Professional development is a tool for 
improving the effectiveness of educators in facilitating student learning (Loucks-Horsley, 
Stiles, Mundry, Love, & Hewson, 2010). Educator professional development programs 
can focus on integration teaching strategies and strategies to minimize or overcome 
integration barriers. However, few effective professional development models exist 
which address integrating science and mathematics (Yasar et ah, 2006). CSM is one 
model which has been developed to both teach science and mathematics teachers to 
integrate and introduce administrators to integration strategies, and it was designed to 
include characteristics of effective professional development (West & Browning, 2010). 
Research suggests that professional development programs that incorporate 
characteristics of effective professional development are correlated with change in 
teacher practices (Binnan, Desimone, Porter, & Garet, 2000; Desimone, Porter, Garet, 
Yoon, and Binnan, 2002; Garet, Porter, Desimone, Birman, & Yoon, 2001; Guskey & 
Yoon, 2009). This thesis will evaluate the overall effectiveness of the 2009-2010 CSM 
program following Guskey ’s (2000) five-level professional development evaluation 
model. 

Professional Development and Evaluation 

The ultimate goal of educational professional development is to increase the 
effectiveness of teachers and other education professionals such that they may better 




6 



facilitate learning for all students (Guskey, 2000, 2002b; Loucks-Horsley et al., 2010; 
National Staff Development Council [NSDC], 2010). That professional development 
positively affects student learning and achievement is a common belief as evidenced by 
the fact that many professional development programs list improving student 
achievement as a primary goal of the program being described or evaluated (for example 
see Buczynski & Hansen, 2010; Johnson & Fargo, 2010; Owston, Sinclair & Wideman, 
2008) and schools routinely incorporate staff professional development in their plans to 
increase student learning. Effective teachers have a positive impact on student outcomes 
(Guskey & Sparks, 1996; Johnson, 2009); therefore if one can increase the quality of 
teachers via professional development, one can improve the learning and achievement of 
students (Buczynski & Hansen, 2010; Johnson & Fargo, 2010; Supovitz & Turner, 2000; 
Yasar et ah, 2006). Yet, because professional development is provided directly to 
educators and only indirectly affects students, it is difficult to measure progress towards 
and assess the achievement of the concurrent goal of improving student learning (Guskey 
& Yoon, 2009). 

Effective professional development. 

The research on professional development has been converging towards a set of 
characteristics that embody effective professional development - professional 
development that ultimately affects student learning and achievement via positively 
affecting teacher and administrator practices and beliefs (Loucks-Horsley et ah, 2010). 

Binnan et al. (2000) describe a framework for effective professional development 
whereby three structural features (activity fonn, duration, and the degree of collective 
participation) impact the success of three core features (the amount of content focus, 
active learning, and coherence with teacher and district goals, beliefs and prior 
professional development experiences) which in turn directly impact how much the 
professional development activity changes teacher practices, knowledge, and skills. This 
framework emerged from a multi-year, in-depth study of the effectiveness of the 
Eisenhower Professional Development Program (EPDP), a source of funding for a 
cornucopia of professional development activities nationwide. 

In addition, several other features characterize effective professional 
development. Professional development should be research-based and standards-based 
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(Guskey & Yoon, 2009; Loucks-Horsley et al., 2010) and build leadership capacity 
(Loucks-Horsley et al., 2010). Developers should consider the context in which the 
program will operate (Guskey, 2009; Loucks-Horsley et al., 2010) and the importance of 
the beliefs of participants on a program’s ultimate success (Guskey, 1991; Johnson, 

2009). 

Activity form. 

Binnan et al. (2000) categorized the form of professional development as either 
traditional activities that take place outside of the regular school day such as workshops 
and institutes, or reform activities that take place during the regular school day such as 
mentoring and study groups. Mixed results have been reported regarding whether the 
professional development form significantly impacts its quality. Garet et al. (2001) 
completed a cross-sectional analysis of 1027 Teacher Activity Surveys documenting six 
months of teacher experiences and behaviors completed by teachers sampled 
proportionally by district size from a national probability sample of school districts 
funded by the EPDP and found that the duration of any professional development activity 
influenced the quality of that activity more than the form of the professional development 
activity did. They note that refonn activities tended to be of longer duration than 
traditional activities and therefore may have developed a reputation as being more 
effective, but traditional activities of the same duration as reform activities had similar 
reported outcomes. Alternatively, Desimone et al. (2002) found in a longitudinal study of 
teacher participants of Eisenhower- funded professional development programs that 
duration had no effect on outcomes while refonn programs that focused specifically on 
higher order skills or strategies for alternative assessment provided significant benefit to 
teacher participants over traditional programs. Although it is unclear whether reform or 
traditional activity forms are differently effective, refonn activities have developed an 
association with being more effective. 

Duration. 

Several studies suggest that the longer the duration (time span and cumulative 
hours) of a professional development program, the more likely the program will be of 
higher quality as there is more time to incorporate active learning, coherence, and 
experiences with content (Basista & Matthews, 2002; Buczynski & Hansen, 2010; Garet 
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et al., 2001; Supovitz & Turner, 2000; Yasar et al., 2006). Supovitz and Turner (2000) 
investigated whether the duration of professional development increased teachers’ use of 
inquiry teaching practices by analyzing data collected as part of the 1997 Local Systemic 
Change Initiative which queried 3263 teachers and 666 principals from two dozen U. S. 
locations about their practices and affectations. They found statistically significant 
correlations between professional development duration and both inquiry teaching 
practices and increased investigative classroom culture; large gains in the amount of 
inquiry teaching and investigative classroom culture occurred after teachers had 
experienced 80 hours and 160 hours of professional development respectively. “Each 
standard deviation of increased content preparation a teacher reported was associated 
with a 20% increase in the use of both investigative teaching practices and investigative 
classroom culture” (Supovitz & Turner, 2000, p. 974). 

Guskey and Yoon (2009) conducted a meta-analysis of research for evidence that 
professional development positively affects student achievement. Nine of 1343 studies 
reviewed met their evidence criteria and they found that the studies with positive 
correlations to student achievement required at least 30 hours of professional 
development contact time. All nine studies concerned elementary teachers and covered 
reading, language arts, science, or mathematics. 

Basista and Matthews (2002) studied science and mathematics professional 
development programs. They note that in their experience, at least 72 contact hours and a 
three week time span is needed to initiate change in teacher practice and beliefs but offer 
no evidence to support their statement. 

In general, the longer the professional development duration, the more effective 
the program. Evidence suggests that at least 30 hours of contact, but more likely a 
minimum of 72-80 hours of professional development contact garners effective 
outcomes. 

Collective participation. 

Collective participation is the degree to which teachers from the same school, 
grade, or department participate together in professional development. Collective 
participation influences the degree of coherence and active learning in the professional 
development activity which impacts change of teacher practices which then influences 
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student learning (Birman et al., 2000; Garet et al., 2001). Desimone et al. (2002) 
conducted a three-year longitudinal study of a purposeful sample of 207 teachers who 
had participated in professional development funded by the Eisenhower Program. They 
established the teachers’ baseline teaching practices during year one, evaluated the 
teachers’ professional development experiences during year two and compared this to 
the teachers’ teaching practices in year three to determine how the structural and core 
features of professional development affect teaching practices. Desimone et al. (2002) 
found that collective participation significantly increases a teacher’s use of a 
technological strategy that was a focus of a professional development program. 

SUNY-Brockport began a multidiscipline institute called Computational Math, 
Science, and Technology (CMST) and in 2003 began to offer ongoing professional 
development in CMST pedagogy to area teachers which included summer institutes, in- 
service workshops, bi-monthly meetings, coaching from university faculty, and student 
competitions (Yasar et al., 2006). Upon evaluation of the program’s impact within the 
Rochester City School District, researchers found a correlation between the number of 
CMST teachers at one school and student achievement such that students taught by 
multiple CMST teachers academically outperformed students taught by only one CMST 
teacher. 

Wright State University ran year-long professional development programs in 
mathematics and science integration for secondary teachers from 1994 through 1999 
(Basista & Matthews, 2002). Each year the university offered a four-week summer 
institute focusing on standards-based integration strategies followed by school-year 
support for teachers and a half-day session for principals. Basista and Matthews (2002) 
evaluated the program’s effects on teacher practices by analyzing classroom 
observations, teacher portfolios, pretests and posttests of content, and surveys. They 
report that after three years of one district’s participation, a participant administrator 
commented “that the student proficiency exam scores in science and mathematics have 
been higher in buildings of the district with large numbers of teacher participants in the 
programs than in buildings that have had fewer participants” (Basista & Matthews, 2002, 
p. 366). 
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Studies of the needs of preservice and in-service teachers have recommended that 
teachers need opportunities to form collaborative relationships and experience with team- 
teaching (Johnson, 2006; Lehman & McDonald, 1988). Collective participation fosters 
in-school collaboration because providing teachers with the same knowledge of a strategy 
or technology can serve as a starting point for further elaboration and sharing of 
classroom experience by those teachers throughout the school year. Depending on the 
professional development activity, collective participation may also allow teachers to 
experience both being taught by a team of instructors and planning and presenting a 
team-taught lesson themselves. 

Additionally, evidence suggests that inclusion of administration in either the same 
or a parallel professional development that teachers participate in may increase the 
chance that teachers will try new strategies in their classrooms. Supovitz and Turner 
(2000), in their study of professional development effects on inquiry-based teaching 
practices, noted that teachers who believed they had administration support utilized 
significantly more inquiry-based practices in their classrooms than teachers who reported 
not having administration support. 

In summary, collective participation of teachers and their administrators is 
associated with more effective professional development. Collective participation leads 
teachers to implement new strategies at higher rates, promotes teacher collaboration, and 
is correlated with increased student achievement. 

Content focus. 

Research suggests that professional development focusing on curriculum content 
improves overall teacher quality (Buczynski & Hansen, 2010; Owston, Sinclair & 
Wideman, 2008; Supovitz & Turner, 2000). Garet et al. (2001) designated content focus 
as a major core feature of professional development. Content focus, in their view, 
includes learning about specific content knowledge, content pedagogy, student learning 
goals per content aspect, and ways students learn different aspects of content. Results 
from their national probability sampled survey of teachers (previously described) 
indicated that professional development that focuses on content positively affects teacher 
knowledge and skills, and surveyed teachers reported that they were more likely to 
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change their teaching practices when their content knowledge and skills had been 
augmented. 

Results from the corresponding longitudinal study previously described also 
indicated that content focus is important. Desimone et al. (2002) learned that teachers’ 
use of a specific strategy in their classrooms did increase as a result of participation in 
professional development that focused “squarely on that specific practice” (p. 91). The 
increase of strategy use occurred regardless of teacher grade level, discipline, or whether 
the teacher had prior experience with the strategy. 

Evaluation results from a Summer Mathematics Institute reported by Hartsell, 
Herron, Fang, and Rathod (2009) serve as an example of this generality. The Summer 
Mathematics Institute ran for four summers from 2005 - 2008, included 20 meeting days 
during the summers with two follow-up days in the fall, and focused on 14 mathematics 
concepts and how to incorporate technology such as Excel, PowerPoint, and graphing 
calculators into mathematics problem solving. Hartsell et al. (2009) analyzed data from 
pretests and posttests of teachers’ level of knowledge of mathematics and technology and 
presurveys and postsurveys which covered the frequency with which teachers used the 
focused technologies to solve mathematics problems, and the teachers’ confidence in 
applying the focused technologies to their mathematics instruction. The researchers 
reported a significant increase in (a) the teachers’ knowledge of Excel, PowerPoint, 
graphing calculators and the focused mathematics concepts, (b) the frequency with which 
teachers used technology to solve mathematics problems, (c) the teachers’ confidence in 
using technology to solve mathematics problems in 10 of the 14 areas of mathematics 
focus and (d) the teachers’ likelihood of applying their new skills in their classrooms. As 
with Desimone et al. (2002), as teachers gained knowledge of a technology or 
mathematics concept, they were more likely to incorporate it into their teaching practice. 

Quick, Holtzman, and Chaney’s (2009) investigation of how professional 
development characteristics affected teacher practices in English Language Arts (ELA) 
implies that professional development that does focus on content pedagogy should also 
focus on the content itself; content pedagogy should not be addressed alone. They 
studied elementary literacy instruction in nine San Diego schools from 2004 through 
2006. During year one, teachers completed professional development logs and were 
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interviewed. During both years one and two, 90-minute classroom observations were 
completed and instruction was coded every five minutes for seven dimensions of 
instruction. Regression of year one’s instructional quality onto year one’s professional 
development experiences showed that professional development that focused on content 
was positively correlated with teachers using high-level questioning and discussion 
strategies; the effect size was 0.28. Professional development that focused primarily on 
content pedagogy had a negative correlation with high-level questioning and discussion 
strategies. When year one professional development experiences were used to predict 
year two instruction, no significant correlations were found, but correlation directionality 
was maintained and professional development that focused on content without content 
pedagogy showed a slightly larger effect than that which focused on both, and showed a 
much larger effect than professional development that focused only on content pedagogy. 

Professional development that focuses on content knowledge is associated with 
increased teacher knowledge and positive change in teacher practice. The professional 
development should focus on specific content or technology goals and if content 
pedagogy is taught, it should be taught concurrently with content knowledge. 

Active learning. 

A common saying is that teachers teach the way they were taught (NCTM, 1989; 
Watanabe & Huntley, 1998). Considering that today’s inquiry, discovery, and 
experiential teaching strategies were put into practice after the time when many teachers 
were students themselves, the majority of teachers may have little experience with and 
possibly low understanding of today’s preferred teaching methods. Active learning 
during professional development gives teachers a chance to be students again as they 
experience curricula as their students should and see modern teaching strategies modeled 
for them. 

Garet et al. (2001) found that professional development that incorporates active 
learning is correlated with increased teacher knowledge and skills. Desimone et al. 
(2002) reported that incorporation of active learning also significantly increases teachers’ 
use of the specific strategy that was the focus of a professional development program. 

Similarly, Foss and Pinchback (1998) evaluated one section of a state -wide 
integration professional development class in Arkansas and found that active learning 
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contributed to teacher content knowledge and change in practice. Foss and Pinchback 
evaluated one of ten integration classes provided to K-4 teachers. The class encompassed 
two semesters, included 50 teachers, and the class instructors team- taught 29 integrated 
lessons in mathematics, science and reading which the teachers experienced as students. 
Data collected included three content pretests and posttests, a materials survey, teacher 
reactions to the class, and teacher class work such as reflections, journals, portfolios, 
thematic units, and lesson plans. The pretests and posttests showed significant increases 
in teacher content knowledge in all three disciplines. Although no classroom 
observations were undertaken by the researchers, they reported that teacher class work 
strongly suggested the teachers were beginning to change their teaching practices to 
incorporate the modules. Additionally, teacher responses continuously noted the 
importance of learning as the students do. 

Wenglinsky and Silverstein (2007) found that teachers who participated in an 
experiential summer science institute changed their teaching practices and the students of 
these teachers demonstrated increased science performance. Silverstein directed 
Columbia University’s Summer Research Program for Secondary School Science 
Teachers which sponsored a dozen teachers each summer and provided significant funds 
for the teachers’ classrooms. The Summer Research Program gave teachers the 
opportunity to work on real-life science problems in a lab with expert scientist mentors 
and during the subsequent school year, provided the teachers with mentor scientist 
support and classroom visits. As part of the program evaluation, Wenglinsky and 
Silverstein found that teacher participants changed their teaching practices to incorporate 
more constructivist principles and more probing and higher-order-thinking questions and 
were more empathetic to student difficulties with content. The passing rate of students of 
participant teachers was 7-8% higher on the New York State Regents Exam than the 
passing rate of students of non-participant teachers. Additionally, students of participant 
teachers were more involved in science than students of other teachers as they were two 
and one -half times more likely to enter a science competition and four times more likely 
to join a school science club. 

Therefore, incorporation of active and experiential learning for participants in 
professional development is associated with increased teacher content knowledge and 
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skills and more constructivist teaching practices. Additionally, improved student 
achievement and interest in science is correlated with active learning in professional 
development. 

Coherence. 

Professional development is coherent if it is aligned with standards and the vision 
of a department, school or district, and makes connections with teachers’ goals, beliefs, 
and previous professional development experiences (Garet et ah, 2001; Loucks-Horsley 
et ah, 2010). In a national survey of teachers, Garet et al. (2001) detennined that 
coherent professional development experiences are positively correlated with changes in 
teacher practice beyond the increase attributed to teacher increased knowledge and skills. 

Teachers are urged to teach the state standards for their grade level and discipline; 
these standards are what the students are assessed on and the students’ performance 
affects the school’s standing. In addition to being aligned with state and national 
standards, Lehman (1994) suggests professional development activities be explicit in 
demonstrating how the activity content connects to the standards and local scope and 
sequences. Lehman surveyed preservice and in-service elementary teachers for their 
perceptions on science and mathematics integration. He concluded that many 
participants were unfamiliar with national standards as the teachers felt integration was 
an add-on topic (integration is promoted by national standards), and many participants 
commented that they did not know where or how integrated lessons would even align 
with their curricula. 

Johnson (2006) considered barriers to implementation of standards-based 
initiative teaching reform in her study with the Ohio Model School Initiative Program. 
She found that many teachers have only a basic understanding of the National Science 
Education Standards (NRC, 1996) and may not be able to connect professional 
development and teaching practices to the standards. Such a finding supports 
professional development clearly linking its content to standards. 

Johnson (2007) investigated what cultural, technical and political barriers teachers 
encountered when implementing standards-based instruction and how such barriers 
impeded teaching practices. Johnson analyzed data from classroom observations, 
interviews and surveys from science teachers of grades six through eight at two suburban 
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Ohio middle schools who were in their second year of participation in a Model School 
Initiative reform. As compared with School One, School Two faced higher political 
barriers than School One, meaning there was a lack of school and district alignment. 
School One’s principal participated in the reform effort while School Two’s principal did 
not and School Two also lost its professional development funding during the course of 
the study. Johnson found that all teachers experience different degrees of each type of 
barrier and teachers with few cultural or technical barriers are able to develop methods to 
work-around political barriers. Administration support of teacher decisions is important 
for minimizing political barriers and is especially crucial for teachers with zero to three 
years of experience as such teachers are stymied by many barriers of all three types. 
Johnson reported that teachers who encountered multiple political barriers exhibited low 
implementation levels of standards-based instruction. 

Teacher beliefs can also be a significant barrier to change in teacher practice 
(Johnson, 2006) and therefore should be considered and accommodated during 
professional development activities to increase the chance that the activity will positively 
affect teaching practices. Johnson (2009) conducted case studies of several science 
teachers who had participated in a three-year professional development program and 
found that the effectiveness of teachers hinged largely on each individual teacher’s 
beliefs. Effective teachers love learning and are open to new ideas while ineffective 
teachers do not see a need to change their teaching practices, may have prior experiences 
that prejudice them against trying new strategies, or may have misconceptions regarding 
the purpose of inquiry. 

Ensuring coherency, or aligning professional development objectives with local 
school goals as well as with teacher prior experiences and individual learning goals, is an 
important step towards effective professional development. To promote change in 
teacher practice, professional development should explicitly link to relevant standards, 
align with school and district goals, and link to teachers’ beliefs to demonstrate how the 
professional development activities are relevant to each teacher. 

Context. 

“Professional learning occurs within people who have extensive experience and 
who live and work in unique contexts that can either thwart or support professional 
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development” (Loucks-Horsley et al., 2010, p. 80). Effective professional development 
is adjusted and re-molded for each setting. Factors that come into play include: the 
learning needs of teachers and students, teaching practices, the learning environment, 
school culture, leadership stance, policies, resources, and the community (Loucks- 
Horsley et al., 2010). 

Johnson and Fargo (2010) described and reported on the positive results of the 
Transformative Professional Development (TPD) Model that integrates the educational 
context of participants with science teaching practices. TPD is a tripartite model that 
focuses on whole-school science professional development in standards-based 
instruction, inquiry and inclusion of literacy into science, building student-teacher 
relations by promoting home-visits and culturally-relevant lessons, and building a 
positive classroom environment. The TPD Model was tested for effectiveness in an 
urban school district over a two-year period. The study matched four middle schools in 
the same district by their state science results such that one school in each pair 
participated in the professional development and the other served as the control. 

Teachers from the participant middle schools were trained in a commercially available 
inquiry curriculum as well as culturally relevant and student-oriented teaching practices. 
Teachers were involved in the professional development direction and initiated 
cooperative learning structures. Effectiveness was measured with teacher observations 
and student pretests and posttests. Results show that all but two TPD participant teachers 
improved their teaching practices with regard to lesson design and implementation, and 
by the end of the two years the students of teachers who participated in TPD greatly 
outperformed students of control teachers who were not involved in TPD. The TPD 
participant teachers had been trained to consider their educational context and specific 
student needs and as a result were better equipped to make learning relevant to their 
students. 

Evaluating professional development. 

Just as teachers are encouraged to continually reflect and revise, so too are 
designers of professional development. Evaluation of professional development is an 
ongoing process that takes place from the informal formative daily review to the fonnal 
summative end-of-program review (Loucks-Horsley et al., 2010). On all points on its 




17 



spectrum, evaluation incorporates “systematic investigation of merit or worth” (Guskey, 
2000, p. 41). 

Guskey (2000) developed a five-level model for evaluating professional 
development which has since become widely utilized. Level one addresses participants’ 
reactions to the professional development as a whole in order to optimize the program’s 
design and operation. Participants’ reactions may be obtained via questionnaires, 
interviews, focus groups, or reflection logs at any time during or after the professional 
development. 

Guskey’s (2000) level two assesses participants’ learning - new knowledge and 
skills, behavior, and/or affects - resulting from the professional development. The 
purpose of this evaluation level is to improve the professional development’s content and 
to discern if it had any impact on the participants’ teaching practices. Data may be 
obtained a multitude of ways in level two such as pretests and posttests, participant 
reflections, participant portfolios, simulations or demonstrations, case studies, or 
comparison groups. In a later publication, Guskey (2002a) suggests that professional 
developers should prepare specific learning goals for participants and definitions of 
participant success prior to implementing the professional development. 

As discussed earlier, administration support and consideration of individual 
school’s context is important for maintaining long-tenn and far-reaching effects of 
professional development; level three evaluates measures that characterize the degree of 
organizational support for or built by the professional development and its induced 
change. Level three data may be obtained from direct observation of teaching and 
administrating, school or district records, meeting minutes, questionnaires, focus groups, 
structured interviews, and participant portfolios. Evaluation of such data can illuminate 
strengths or weaknesses of the professional development to induce cultural change, 
barriers to implementation of teaching practices advocated by the professional 
development, or identify future areas of focus. Organizational support is important and 
its omission can impede or prevent the professional development’s success (Guskey, 
2000, 2002a). 

Level four (Guskey, 2000, 2002a) assesses professional development’s main 
direct effects - its participants’ use of new knowledge and skills. Level four evaluation is 
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important because “if there are no notable differences between the professional practices 
of educators using the newly acquired knowledge and skills and those of other educators 
who were not involved, differences in student learning are unlikely” (Guskey, 2000, p. 
187). Evaluators should consider measurement timing for level four as some skills 
require percolation time for participants to internalize new practices before the skill’s use 
is apparent. Level four results indicate what the program is doing well and what needs 
improvement regarding the professional development core features of content knowledge 
and active learning. Data sources include questionnaires, structured interviews, 
participant reflections, participant portfolios, direct observation, tapes, and use of 
comparison groups. Guskey (2000) stresses and Loucks-Horsley et al. (2010) concur that 
effective professional development plans with the end goal in mind such that the type and 
timing of level four assessments as well as definitions of success are considered prior to 
implementation. 

The final level, five, assesses a professional development program’s indirect 
effect on student learning and achievement. The level five analysis provides a picture of 
a program’s overall effectiveness and results can be used to improve the design or 
implementation. Data comes from pretests and posttests, comparison groups, student 
records, school records, questionnaires, structured interviews and participant portfolios. 
As with level four evaluation, the timing of level five assessments is also important. 
Additionally, because so many factors influence student learning, the measurements at 
level five should be reliable and valid, properly sampled, and cross-checked with multiple 
measures (Guskey, 2000, 2002a). 

Guskey’s (2000) five-level evaluation model for professional development 
provides a framework for evaluating many aspects of a professional development 
program. This thesis will follow Guskey’s model to evaluate the effectiveness of the 
CSM professional development program during 2009-2010. A variety of data from 
participants and participants’ students will demonstrate the degree to which CSM met its 
program goals, aligned with accepted standards for effective professional development, 
and enhanced teacher and student outcomes. 
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Rationale for Science and Mathematics Integration 

Integration of science and mathematics has been of interest to educators and 
researchers for a century (Berlin & Lee, 2005; Hurley, 2001). Since the late 1980’s 
national professional and scientific organizations have called for integration of science 
and mathematics in public education, and collaboration of teachers between and across 
disciplines (AAAS, 1989, 1993; NCTM 1989, 2000; NRC 1996). National and many 
state science and mathematics standards have since been revised to incorporate 
integration of science and mathematics processes and content (Berlin, 1994; NCTM, 
1989, 2000; NRC, 1996; TEA, 2010). Specifically, the National Science Education 
Standards (NRC, 1996) states that the “coordination of the science program with 
mathematics education” (NRC, 1996, p. 7) is a necessary condition for development of a 
quality school program. The Principles and Standards for School Mathematics (NCTM, 
2000) includes a connections strand at every grade level which stresses that students 
should recognize and understand how mathematics ideas are related both to other 
mathematics concepts and to concepts of other disciplines, especially science. Recently 
revised Texas Essential Knowledge and Skills (TEA, 2010) emphasize at all grade levels 
that “connections within and outside mathematics . . . underlie all content areas in 
mathematics” (TEA, 2010, § 1 1 1 .24 A. 3) and that “recurring themes are pervasive in 
sciences, mathematics, and technology [that] . . . transcend disciplinary boundaries” 
(TEA, 2010, §112.18 A.3). 

Theoretical support for integration. 

The educational theory supporting integration is strong and has been argued for 
and summarized many times over. Science and mathematics are closely related fields of 
inquiry that seek to understand the natural world (Berlin & White, 1992; McBride & 
Silverman, 1991; Watanabe & Huntley, 1998). The disciplines share process and 
thinking skills (Berlin & White, 1994; NCTM, 2000). Concepts of one discipline are 
used to deepen understanding of concepts of the other discipline (Berlin & White, 1992; 
McBride & Silverman, 1991). Often this concept of cross-pollination is decried as 
science using mathematics as a tool and mathematics using science for relevant 
applications (Watanabe & Huntley, 1998; West & Browning, 2010). However, the 
connection is more complex than that one description and such use does elevate student 
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interest and motivation in class (Furner & Kumar, 2007; McBride & Silverman, 1991; 
Watanabe & Huntley, 1998). At the current stage of discovery, one might argue that 
science and mathematics could not exist without the other and, in fact, many combined 
sub-disciplines have emerged such as geological engineering, applied mathematics, 
computer science, biological engineering, and modeling. Such merging of disciplines 
aligns with the continuum models of science and mathematics integration that have 
perpetuated and evolved through time (Huntley, 1998; Lonning & DeFranco, 1997; West 
& Browning, 2010). Scientists at the NSF/SSMA Wingspread Conference also 
advocated integration as a teaching strategy to bridge students from concrete to abstract 
representations of concepts (Berlin & White, 1992). 

Empirical support for integration. 

Although vast theoretical support and general advocacy for integration exists, 
relatively little empirical evidence exists that demonstrates positive effects of integrated 
instruction on student learning (Berlin & Lee, 2005). Researchers have repeatedly 
lamented this lack of empirical data (Childress, 1996; Frykhohn & Glasson, 2005; 
McBride & Silverman, 1991; Pang & Good, 2000; St. Clair, Hough, & Southwest 
Missouri State University, 1992; West & Browning, 2010; West, Tooke, & Muller, 2003; 
Westbrook, 1998) and express concern that integration of science and mathematics is still 
not widely implemented regardless of the data (Merrill, 2001). Below is a summary of 
positive correlations of integration with student learning and achievement. 

Hurley (2001) completed a meta-analysis of science and mathematics integration 
studies published between 1935 and 1997 that dealt with grades K - college. Thirty-one 
studies were identified that fit her criteria for empirical data which included that (a) the 
studies used control and treatment groups, (b) the treatment group incorporated integrated 
science and mathematics, (c) group sizes were not small, (d) groups were equivalent 
pretreatment, (e) the study measured student achievement in science or mathematics, and 
(f) the study gave enough data for Hurley to calculate an effect size. Hurley calculated 
Cohen’s d effect size for both science and mathematics achievement and summarized 
results by historical educational reform period. Hurley’s analysis demonstrated that on 
average, integration of science and mathematics does have a small positive effect on 
students’ achievement in both science and mathematics, and that during the 1960’s- 
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1 970’s era of curriculum improvement projects, integration had a medium positive effect 
on students’ achievement in science and mathematics. A summary of Hurley’s effect 
sizes is shown in Table 1. 

Table 1 

Effects of integration on students' achievement by reform period as calculated by Hurley 
( 2001 ) 





Dominant 
reform type 




Science 






Mathematics 


Decade 


n 


Cohen’s d 


Effect size 


n 


Cohen’s d 


Effect size 


1930’s - 
1950’s 


Core curriculum 
studies 


5 


0.27 


small 


9 


0.15 


small 


1960’s - 
1970’s 


Curriculum 

improvement 

projects 


6 


0.56 


medium 


10 


0.57 


medium 


1980’s - 
1990’s 


Integration 

programs 


10 


0.30 


small 


10 


0.07 


small 


1930’s - 


All decades 


21 


0.37 


small - 


29 


0.27 


small - 


1990’s 


combined 


medium 


medium 



Although it is not integrated science and mathematics in the strictest sense, 
research on incorporation of environmental education into every subject showed a 
positive effect on several skill sets (Bartosh, Tudor, Ferguson, & Taylor, 2009). 
Environmental education is required by several states to be integrated in all subjects at all 
grade levels. Bartosh, Tudor, Ferguson, and Taylor (2009) compared 8 th grade student 
achievement on a Pacific Education Institute environmental education test which focused 
on inquiry, interrelationships of natural and social systems, and civics, and the 
Washington Assessment of Student Learning (WASL) between five schools the authors 
ranked high in degree of environmental education and five schools they ranked low in 
degree of environmental education. Students from high-ranked schools performed 
significantly better than students from low-ranked schools on the civics and systems 
sections of the Pacific Institute assessment with medium Cohen’s d effect sizes of 0.4 and 
0.5 respectively. On the WASL, students from high-ranked schools also performed 
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significantly better than students from low-ranked schools on the mathematics, writing, 
reading, and listening sections. Small effect sizes were calculated for mathematics (0.3), 
reading (0.2), and listening (0.2) and a large effect size was obtained for writing (0.8). 

Judson and Sawada (2000) reported on action research by one 8 th grade science 
teacher which demonstrated a positive correlation between integration of statistics into 
science class and student achievement in mathematics class. The statistics unit test 
results of two academically similar 8 th grade mathematics classes with the same 
mathematics teacher were compared after one class received integrated science lessons 
from one science teacher, and the other class received non-integrated science lessons 
from a second science teacher. The integrating science teacher had recently participated 
in an integration professional development program and conferred regularly with the 
mathematics teachers during the statistics unit such that the science teacher incorporated 
different methods of data collection and display into the regular science lessons. As 
compared against grades of students in the non-integrated science class, the grades of the 
students in the integrated science class were significantly higher on the statistics unit test 
(X (4, n=52) = 16.92, p<0.05) and remained similar in science. 

Westbrook (1998) investigated how 9 th grade students’ conceptual understanding 
of density and graphing as measured by analysis of student-created concept maps were 
affected by integrated teaching. Westbrook compared concept maps of students in one 
integrated algebra and physical science class with academically similar students in non- 
integrated physical science who were also concurrently enrolled in algebra. Students 
created concept maps before, during and after a unit on density, graphing, and slope. 
Students in the integrated class were team-taught by a science and a mathematics teacher 
and were led by the mathematics teacher through a discussion of the connections between 
density and slope during the unit’s graphing phase unlike the students in the non- 
integrated science class who also worked on density graphs but only received direct 
instruction on how to use certain mathematical equations to calculate graph parameters. 
Westbrook determined that although both groups of students demonstrated an 
understanding of the main concepts of the unit and both groups used relational linkages 
equally as frequently in their concept maps, students in the integrated class demonstrated 
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a deeper understanding of unit concepts by including more procedural linkages within 
their concept maps. 

As the first part of a project to develop assessments for the integrated Activities in 
Math and Science (AIMS) curriculum, Berlin and Hillen (1994) uncovered benefits of the 
AIMS program to student behaviors and affects. A total of 2025 4 th , 5 th , and 6 th grade 
teachers from six states were recruited to participate in the study. Both teachers and 
trained classroom observers collected data on student behavior for two months. Many 
positive student outcomes were identified and categorized into three groups. Students 
developed better attitudes and behaviors toward themselves such as improved listening 
skills, time management, self-control and work habits. Students developed improved 
relationships with their peers as evidenced by increased peer encouragement, empathy 
and the respecting of opinions. Finally, students developed improved interactions with 
both peers and adults as evidenced by increased self-resolution of group conflicts, and 
increased collaboration and development of leadership skills, among other skills. 

However, two studies of science and mathematics integration published after or 
not included in Hurley’s (2001) meta-analysis did not report significant differences in 
student performance as compared with students in comparable non-integrated settings. 
Childress (1996) evaluated one activity entitled Capture the Wind from the NSF- 
supported Technology, Science, Mathematics Integration Project (TSM) as implemented 
with two 8 th grade technology classes, one of which served as the control and one of 
which received integrated lessons during the module. Childress found no significant 
difference in student performance on a posttest but did note that during interviews, 
students in the treatment (integrated) class discussed how they attempted to incorporate 
knowledge obtained during the integrated science and mathematics portions to their wind 
collector re-design. 

Merrill (2001) conducted a quasi-experiment of the effects of integrated 
technology, science and mathematics instruction on high school student performance 
during a two-week unit on energy, power, energy efficiency and mechanical advantage. 
Both control and experimental groups received the same content lessons from the same 
teacher. The control class completed worksheets for content elaboration while the 
experimental class worked with a Pedal 4 Power Energy Education Bicycle. Analysis of 
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pretests and posttests, of which there were three at two-week intervals, showed no 
significant difference in student achievement between groups. Both groups demonstrated 
similar cognitive gains and maintained those gains over time, and both groups were 
equally able to identify integrated terms and phrases. However, Merrill was not clear 
whether the content was taught differently between classes, therefore, this may not be a 
true integration experiment, but may only be a comparison of traditional versus hands-on 
enrichment. 

Considering that integration has been researched for a century and is apparently 
important as evidenced by national standards, this list of empirical studies is relatively 
small. Indeed, indications are that integration has at least a small effect on student 
achievement but half of the above studies do not provide enough data to be included in an 
updated integration meta-analysis similar to Hurley (2001). Additional rigorous studies 
of the effects of integration of science and mathematics on student achievement are 
warranted to confirm existing trends and extend the validity to other contexts. 

Successful integration professional development. 

There appear to be even fewer reports of effective professional development 
models which provide integration teaching strategies to teachers and also report on 
student outcomes. The professional development program CMST (Yasar et ah, 2006), 
previously described in the Collective Participation section of this paper, is a successful 
professional development model for integration. CMST is an outgrowth of SUNY- 
Brockport’s Computational Science (CPS) degree program which blends mathematics, 
science and technology and focuses on modeling and problem solving. CMST utilizes 
the skills and resources from eight university departments and two area school districts to 
provide the CMST summer teacher institutes and school year coaching. Teachers are 
also provided with technology equipment and have access to hundreds of CMST lessons 
online. The researchers found a strong correlation between student performance and 
increasing amounts of CMST training for teachers. Additionally, student attendance 
increased and dropout rates decreased with increased exposure to CMST-based 
instruction. The CMST program also sponsored an annual Physics Week and Challenge 
Competition; students who participated in the Challenge Competition academically 
outperformed students from the same school who did not. Additional studies on the 
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effectiveness of integration professional development on student outcomes are needed to 
corroborate and extend the results of the study. 

Barriers to integration. 

Many researchers have reported that teachers encounter a wide variety of barriers 
that hinder their employment of or prevent them from trying integration teaching 
strategies. Johnson (2006, 2007) grouped such barriers into three categories - political 
barriers, technical barriers, and cultural barriers. Political barriers are often obstacles that 
teachers do not have direct control over and are, to a degree, strong outside influences. 
Political barriers include: (a) lack of support, flexibility or openness from administration 
and department leaders (Huntley, 1999; Johnson, 2006, 2007); (b) lack of or limited 
resources, especially of science materials for use by mathematics teachers (Buczynski & 
Hansen, 2010; Huntley, 1999; Johnson, 2006, 2007; Johnson & Fargo, 2010; Lehman, 
1994; McBride & Silverman, 1991); (c) limited access to in-service professional 
development (Johnson, 2006, 2007); and (d) pressure to confonn to the mandated 
curriculum (Buczynski & Hansen, 2010). School structure, classroom issues and teacher 
skill sets are classified as technical barriers and as such, teachers may be able to apply 
changes to their practices to minimize some technical obstacles. Technical barriers 
include: (a) curriculum scope and sequences that are ill-defined or do not readily 
accommodate integrated teaching practices (Huntley, 1999; Lehman, 1994); (b) the 
structure of the school day and the temporal and physical separation of disciplines within 
the school (McBride & Silvennan, 1991); (c) lack of teacher between-discipline and 
cross-discipline planning and collaboration time (Johnson, 2006, 2007); (d) limited 
teacher content knowledge and confidence, especially of subjects outside their primary 
discipline (Frykhohn & Glasson, 2005; Furner & Kumar, 2007; Johnson, 2006, 2007; 
Wantabee & Huntley, 1998); (e) the lack of teacher experience with integration 
(Frykhohn & Glasson, 2005); and (f) the limited number of integrated curriculum models 
and the lack of teacher awareness of existing models (Huntley, 1999; McBride & 
Silverman, 1991). Additionally, integrated and inquiry-based lessons take more time 
than traditional teaching methods and the school day structure constrains instructional 
time (Buczynski & Hansen, 2010; Johnson, 2006, 2007; McBride & Silverman, 1991; 
Wantabee & Huntley, 1998). Cultural barriers have to do with prior experiences and 
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individual characteristics of teachers and include obstacles such as teachers’ beliefs and 
values which may hinder teachers’ ability to adapt new practices (Johnson, 2006, 2007; 
Wantabee & Huntley, 1998). Personal reflection may be required for individual teachers 
to recognize and overcome cultural barriers. 

What do teachers require for integration to occur? 

In order for integrated teaching practices to take hold and barriers to integration to 
be overcome, teachers require certain knowledge and skills from professional 
development experiences. Teachers have limited understanding of and experience with 
integration and would benefit from a clear definition of integrated curriculum as well as 
experience with integration strategies (Frykhohn & Glasson, 2005; Lehman, 1994; 
Lehman & McDonald, 1988; Stinson, Harkness, Meyer & Stallworth, 2009). Integrated 
lessons require knowledge of at least two disciplines and teachers may require additional 
content knowledge in their non-primary discipline (Berlin, 1994; Frykhohn & Glasson, 
2005; Furner & Kumar, 2007; Stinson et al., 2009). Finally, teachers require quality 
integrated curriculum models (Lehman & McDonald, 1988; McBride & Silverman, 1991; 
Pang & Good, 2000). 

Integration definition and experience. 

The definition of integration remains disputed amongst researchers (Berlin, 1994; Berlin 
& Lee, 2005; Berlin & White, 1992) and teachers are also confused about what 
integration really means and looks like in a classroom (Stinson et al., 2009). Berlin and 
Lee (2005) performed a historical analysis of integration publications from 1901 through 
2001 and found that integration tenninology was inconsistent and therefore research 
results were difficult to compare. Hurley (2001) conducted a historical review and meta- 
analysis of integration research and concluded that integration exists in multiple forms. 
Hurley classified the research into five categories which take into account both content 
and sequence (see Figure 2). There was no correlation between decade of publication, as 
a proxy for a time period’s predominant educational philosophy on integration, and 
integration category. Current accepted integration models promote integration as a 
continuum (Huntley, 1998; Lonning & DeFranco, 1994; Lonning, DeFranco, & 
Weinland, 1998; West & Browning, 2010) of content and may be interpreted as sub- 
parallel to Hurley’s (2001) categories. 
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Greatest 

Total: disciplines are taught together, equally 

Enhanced: one discipline is the instruction focus, while the second discipline is 
visible throughout the lesson 

Partial: disciplines are taught in the same class and are sometimes treated 
concurrently and sometimes treated separately 

Parallel: disciplines are taught simultaneously with related concepts 

Sequenced: disciplines are taught sequentially with related concepts 

Least 

Figure 2. Levels of Integration. Each level summarizes when each discipline is taught 
during integrated lessons and the degree of integration. Adapted from Hurley (2001). 

Teachers, who as a whole may be characterized as application-oriented consumers 
of research on integration, likely are not experts in integration definition nuances and are 
likely to develop their own (incomplete) conceptions of integration. Stinson et al. (2009) 
investigated the perceptions teachers have of integration via a survey which asked 
respondents to categorize integrated teaching scenarios as integrated or not and also 
asked for respondents’ understanding of integrated mathematics and science. 

Scenarios that contained content and activities that were arguably more common 
(i.e., solar system, graphing) tended to be identified more frequently as integrated. 
Scenarios that were less explicit in conveying content or activities (i.e., 
investigating crystal formation, studying historical earthquakes) tended to be 
identified as not integrated. (Stinson et ah, 2009, p. 159) 

The researchers reported that teachers require information in the instructional context in 
order to detennine whether a lesson is integrated and a teacher’s depth of content 
knowledge affects how readily that teacher recognizes integration. 

Huntley (1999) reported on a case study which exemplifies how teachers can 
misinterpret integrated teaching. For two non-consecutive weeks, Huntley observed a 
middle school mathematics and science teacher pair who self-reported that they were 
completely integrating science and mathematics instruction, then interviewed the teachers 
and also analyzed the teachers’ self-reports. Huntley determined that the pair were not in 
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fact integrating but instead were teaching interdisciplinarily. The two teachers combined 
their class periods and rotated the students through work styles or learning stations. 
Huntley, utilizing her Mathematics/Science Continuum (Huntley, 1998), categorized the 
teachers’ fundamentals section where students drilled and practiced mathematics skills as 
mathematics for the sake of mathematics, the integrated stations where students rotated 
through individual, group and teacher-led activities as interdisciplinary since each station 
was either mathematics with science or science with mathematics, and the research 
section where students were led through science investigations as science with 
mathematics. Much of the instruction was teacher-driven and each section was clearly 
identifiable as either mathematics or science. 

Teachers teach as they were taught (Watanabe & Huntley, 1998). Since many 
current teachers did not learn via integrated curriculum, teachers would benefit from 
experiencing integrated lessons themselves (Frykhohn & Glasson, 2005; Lehman, 1994; 
Lehman & McDonald, 1988). Lehman and McDonald (1988) surveyed preservice 
teachers who were taking a required integration class before and after the class for their 
perceptions of integration. The percentage of preservice teachers reporting that they had 
experienced integrated lessons during their schooling dropped significantly from 73% to 
13% after learning about integration teaching strategies. In-service science and 
mathematics teachers were also surveyed for their perception of integration. A large 
percentage of teachers (79% of science teachers and 42% of mathematics teachers) 
responded that they often integrate their lessons but only 32% of science teachers 
indicated awareness of available integrated curriculum materials while 42% of 
mathematics teachers did. Based on these discrepant survey results Lehman and 
McDonald suggested that in-service teachers over-reported their use of integration 
strategies and did not fully understand what integration means. This conclusion aligns 
with the later suggestion by Stinson et al. (2009) that science teachers do not comprehend 
integration as researchers do. 

Teachers do not hold complete conceptions of integrated lessons and are not able 
to identify lessons as integrated when lesson concepts increase in complexity. As 
Lehman and McDonald’s (1988) survey suggest, providing teachers with integrated 
strategies and experience with integrated lessons helps to define integration for teachers 
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and starts to align teachers’ conceptions of integration with those that are dominant in the 
research. 

Content knowledge. 

Most teachers require additional content knowledge (Stinson et al., 2009). Since 
integration combines two disciplines, teachers already weak in their own discipline likely 
lack the content knowledge and/or confidence to integrate a second discipline (Berlin, 
1994; Furner & Kumar, 2007; Frykhohn & Glasson, 2005). “Teachers with limited 
understandings of mathematics and science likely have limited capacity for making deep 
connections between the disciplines during classroom instruction” (Huntley, 1999, p. 64). 
Additionally, starting in the 1990’s, corresponding to initiation of standards-based 
initiatives, the grade-level focus for integration has switched from elementary to 
secondary lessons (Berlin & Lee, 2005). While many elementary teachers are self- 
contained and teach all subjects to the same small group of students, secondary teachers 
generally teach one discipline to numerous students, do not have the opportunity or 
collaboration time to team-teach or discuss cross-discipline connections, and likely lack 
the depth of knowledge in the second discipline to adequately prepare and lead an 
integrated lesson (Frykhohn & Glasson, 2005; Johnson, 2006; McBride & Silvennan, 
1991). 

With additional knowledge and experience, elementary teachers and secondary 
teachers are able to develop integration skills. Frykhohn and Glasson (2005) investigated 
how instruction in science and mathematics connections affected the teaching practices of 
preservice teachers during student teaching. The project, conducted over two years with 
two cohorts of student teachers, instructed participants to work in collaborative groups to 
create integrated science and mathematics activities that would be field-tested during 
student teaching experiences. Prior to project initiation, student teachers expressed 
apprehension regarding their lack of content knowledge in their non-primary discipline 
and reluctance to work with others and expose their discipline-naivete. During and after 
project development and lesson teaching, student teachers expressed surprise at how 
often they were able to use science within the context of mathematics or vice versa, but 
they also noted the importance of making connections, having the proper content 
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knowledge and content pedagogy, and collaborating with colleagues of alternate 
disciplines. 

Curriculum models. 

Teachers also need good integrated curriculum models so integrated lessons can 
be more easily incorporated into the classroom without significant development output 
from the teacher (Lehman & McDonald, 1988; McBride & Silverman, 1991; Pang & 
Good, 2000). In a survey of preservice and in-service elementary teachers, less than half 
were aware of integrated curriculum materials and although they understood the 
importance of integration, many in-service teachers felt there was not enough time to add 
integration onto their existing curricula (Lehman, 1994). West, Tooke, and Muller 
(2003) suggest providing teachers with pre-linked (correlated) standards between science 
and mathematics so teachers can more readily see how portions of one discipline can 
support and enhance portions of the other. 

Correlated Science and Mathematics Professional Development Model 

Correlated Science and Mathematics (CSM) is a professional development model 
designed to teach teachers how to integrate science and mathematics in the classroom 
(West & Browning, 2010). CSM is standards-based and aims to give participants a 
conceptual understanding of both science and mathematics content, a framework of how 
the two disciplines can be linked and how the language of the two disciplines is similar 
and different, and experience with inquiry and correlated lessons. 

Development and description. 

The idea for CSM arose from West and Tooke’s (2001) efforts to correlate the 
Texas Essential Knowledge and Skills (TEKS) for science with the mathematics TEKS 
for grades K-5. After Texas revised its standards in the mid- 1 990 ’s, science was de- 
emphasized such that students were no longer required to take a high-stakes science test 
at the end of fourth grade. As a result, less science was taught in K-5 classrooms. West 
and Tooke thought correlated TEKS might help elementary teachers incorporate science 
back into daily lessons. West, Tooke, and Muller (2003) expanded the correlation of 
science and mathematics TEKS to grades 6-8 and suggested the correlations could serve 
as a starting point for integration as many aspects of the science and mathematics 
curricula now were linked. Finally, Vasquez-Mireles and West (2007) described steps 
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that science and mathematics teachers could follow to collaborate and develop correlated 
lessons; a correlated lesson is aligned to both science and mathematics standards and 
teaches parallel science and mathematics content to an equal degree while also using 
proper discipline language. 

The CSM professional development model was first proposed by West, Vasques- 
Mireles, and Coker (2006) as a modular program that would emphasize science and 
mathematics content and pedagogy and provide participants with feedback on integrated 
lessons prepared during the program. CSM was first implemented during the 2006-2007 
school year as a two-week summer institute for teachers with follow-up Saturday sessions 
throughout the school year (Gloyna, 2008). The basic format has evolved to additionally 
include observations of teachers during the school year, inclusion of principals in parallel 
CSM training at the summer institute for two days and at all follow-up Saturday sessions, 
interviews with the teachers and principals during the school year, pretesting and 
posttesting of participant teachers’ students for content knowledge and the use of control 
teachers and their students for comparison of student achievement. 

Offered CSM modules have varied each year. The first 2006-2007 cohort 
participated in Correlated Physics and Mathematics (Gloyna, 2008). Cohort two attended 
during 2007-2008 and participated in Correlated Physics and Mathematics as well as 
Correlated Quantitative Reasoning and Science, and cohort three also attended during 
2007-2008 but participated in Correlated Chemistry and Mathematics alongside 
Correlated Algebraic Reasoning and Science (Gloyna, 2008). Cohort four attended 
during 2008-2009 and participated in Correlated Space Science and Mathematics together 
with Correlated Geology and Mathematics (Duran, 2010). Cohort five, the focus of this 
study, attended during 2009-2010 and participated in Correlated Physics and 
Mathematics concurrently with Correlated Mathematical Reasoning and Science. 

CSM goals. 

The CSM professional development model has five overall program goals. CSM 
strives to (a) increase teacher content knowledge of science and mathematics, (b) 
improve the content pedagogy skills of participant teachers, (c) increase the degree of 
integration in science and mathematics classrooms, (d) improve the perfonnance of 
students of participant teachers in science and mathematics, and (e) improve participant 
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principals’ knowledge of and abilities to evaluate science and mathematics inquiry 
lessons (West, 2009). 

These goals along with the structure of CSM align with characteristics of effective 
professional development previously described (Birman et ah, 2000; Guskey & Yoon, 
2009; Loucks-Horsley et ah, 2010) and national standards for science and mathematics 
curriculum (AAAS, 1989, 1993; NCTM, 1989, 2000; NRC, 1996; TEA, 2010). CSM is 
both an intensive and long-term professional development program as participants attend 
a concentrated two-week summer institute and are supported through the year with five 
AY sessions, two teacher observations, and access to researcher consultation via email. 
CSM requests the collective participation of several science and mathematics teacher 
teams from the same school along with their principals. The summer institute focuses 
intensely on science and mathematics content, content pedagogy, and how the two 
disciplines converge, diverge and can be correlated, and participants are given 
opportunities to experience correlated lessons and inquiry from the student point of view. 
Individual school and teacher contexts are also accommodated as participants are 
encouraged to share experiences and discuss problems and CSM researchers regularly 
communicate with and observe participants throughout the CSM program. 

CSM learner goals or facets. 

Correlated lessons are unique because they seamlessly blend concepts and 
language of two related disciplines into one coherent lesson. Although it is possible for 
individual teachers to teach correlated lessons, West and Browning (2010) suggest that 
CSM may better fit a professional development model because it “is ideally team taught 
by a mathematics expert and a science expert who are well versed in both content and 
content pedagogy” (p. 12). In both cases however, there are several characteristics 
necessary for a lesson to be considered truly correlated. These are delineated by the 
learner goals of CSM professional development (to be referred to as facets): 

(1) develop a conceptual understanding of mathematics and science; (2) use each 
discipline’s proper language; (3) make the natural links between the disciplines; 
(4) identify the parallel ideas between the disciplines; (5) identify language that is 
confusing to students; (6) use standards-based learning objectives; and (7) use a 
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5E inquiry format in science and mathematics when appropriate. (West & 

Browning, 2010, p. 5) 

Incorporation of these seven facets into CSM is supported by research on 
characteristics of effective professional development in general, integration, and effective 
science strategies. Facets one and six are characteristics of effective professional 
development discussed previously. Facets three and four are components of integration, 
also discussed previously. Research supporting facets two and five on language, and 
facet seven on inquiry will be briefly summarized next. 

CSM is unique in emphasizing the use of proper discipline language and 
identification of language that is confusing between disciplines. Other researchers have 
identified the importance of proper language and that confusing language serves as a 
barrier to successful integration (Austin, Converse, Sass, & Tomlins, 1992; Buczynski & 
Hansen, 2010; Oyoo, 2009), but little has been done to promote proper use or overcome 
the language barrier. 

Buczynski and Hansen (2010) studied the effectiveness of inquiry in science 
classrooms on student learning. California Math and Science Partnership (CaMSP) 
professional development provided participant teachers of 4 th - 6 th grade with summer 
sessions on inquiry and school year Saturday sessions on science content and pedagogy. 
Data were collected from teacher pretests and posttests, teacher surveys and focus groups, 
classroom observations, and student scores on the Science California Standards Test. 

One finding that arose was the importance of science vocabulary. Teachers were 
observed mispronouncing or inconsistently pronouncing science vocabulary and 
accepting weak explanations for tenninology from students. The authors suggested the 
teachers’ incomplete understanding of the tenninology and its importance to student 
learning, especially to English-language-learners, was a barrier to successful 
implementation of inquiry because it stymied student understanding and took time away 
from inquiry activities. 

Oyoo (2009) presents a second example of teachers’ lack of comprehension of the 
importance of using and explaining proper or confusing language and its effect on student 
understanding. He studied whether physics teachers in Kenya were aware that some 
science vocabulary is context-dependent and if so, what strategies the teachers used in 
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class to clarify science vocabulary. Oyoo observed nine physics teachers of students at 
the level equivalent to grade 1 1 in the United States in their classrooms, conducted 
teacher and student interviews, and gave students a vocabulary test. Analysis revealed 
that most teachers spent little time on vocabulary as they were rushed to get through 
material, felt that students should have picked up the vocabulary from the context or prior 
science classes, and attributed students’ difficulty with physics to the content and not to 
the vocabulary. Only two experienced teachers made efforts to address vocabulary. 
Teachers erroneously assumed that students proficient in the language had no difficulty 
with science vocabulary. Additionally, teachers were not aware that non-technical words 
could have science-specific meanings that need to be made explicit to students. Although 
this study was carried out in Kenya which has different teacher preparation and education 
expectations than the United States, the few studies summarized here suggest that 
overlooking the importance of science vocabulary is also a problem in the United States. 

Finally, Austin, Converse, Sass, and Tomlins (1992) described differences in 
language usage between mathematics and science lessons that emerged during their two- 
semester professional development for secondary science and mathematics teachers on 
coordinated teaching units. During the fall semester, teacher teams created coordinated 
science and mathematics units that met standards of both disciplines; during the spring 
semester, teachers field tested the units and discussed the results. The group found that 
science and mathematics teachers, textbooks, and students use mathematics and 
vocabulary differently. Between science and mathematics, graphing follows different 
rules, variables are represented differently, slope is used in different contexts, and 
scientific notation is inconsistently represented and uses different mathematical bases. 
Teachers were unaware of the proper usage of language and process in the other 
discipline and the authors suggested that such language discrepancies lead students to 
consider science and mathematics as completely separate disciplines. 

Facet seven calls for correlating teachers to use the 5E inquiry format when 
appropriate. This aligns with national and state standards which recommend the use of 
inquiry in teaching science and mathematics content (NCTM, 2000; NRC, 1996; TEA, 
2010). Although the 5E model is not exactly equivalent to inquiry, Wilson, Taylor, 
Kowalski and Carlson (2010) support West and Browning’s (2010) decision to use the 5E 
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model as a proxy for inquiry as they also did so in their research on inquiry. These 
researchers investigated the effectiveness of inquiry teaching on student learning and 
whether inquiry affects different student demographic groups differently. Wilson et al. 
(2010) conducted a randomized control study where 58 students aged 14-16 years 
attended a 14-hour summer class on Sleep, Sleep Disorders, and Biological Rhythms. 
Participants were compensated upon study completion. The treatment group was taught 
via inquiry and the control group was taught via commonplace strategies such as formal 
presentations, asking students open-ended questions, and teacher-directed group work 
and experiments. Both groups were taught by the same teacher. Students completed 
pretests and posttests and surveys, students were interviewed, and the classes were 
observed. Classroom observation and student survey analysis confirmed that the 
treatment class spent significantly more time using inquiry than the control class (t(48) = 
9.937, p < 0.01 and t(55) = 3.195, p < 0.01, respectively). Posttest scores were controlled 
for the variance in pretest scores and demonstrate that treatment students taught via 
inquiry performed significantly higher than control students taught by commonplace 
strategies (f(l,55) = 4.570, p < 0.05). Regarding demographic differences, the authors 
found that larger white/non- white and male/female perfonnance gaps existed for the 
control group than the treatment group. Inquiry teaching via the 5E model increased 
student learning as well as decreased achievement gaps between demographic groups. 

How does CSM differ from other professional development models? 

The CSM professional development model is a more comprehensive approach to 
teaching participants to correlate and integrate science and mathematics lessons. As 
compared with SUNY-Brockport’s CMST (Yasar et al., 2006) professional development, 
CSM has a similar format, provides content instruction and also models lessons. CSM 
however incorporates principal training and control classes of students of non-participant 
teachers while CMST does not. CMST directly contacts some students through their 
Challenge program, while CSM does not. 

The integrated science and mathematics professional development program of 
Basista and Mathews (2002) had similar goals and format as CSM. Basista and 
Mathews’ program offered teachers a team-taught, standards-based program on science 
and mathematics content, pedagogy, and connections and offered administrators a half- 
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day session on integration and inquiry. While Basista and Mathews did evaluate how 
their professional development affected participant teachers, they neither evaluated 
program effects on the participant administrators, nor measured student learning. 

The Transfonnative Professional Development (TPD) implemented by Johnson 
and Fargo (2010) addressed a different problem than CSM as it sought to improve 
science education at disadvantaged urban schools by developing “culturally relevant 
science and effective teaching strategies” (p. 9). TPD was a longitudinal study that 
followed a similar fonnat and evaluation protocol as CSM and student achievement was 
measured. TPD was standards-based and included lessons on inquiry, like CSM, but 
focused primarily on culturally-relevant teaching strategies and how to integrate literacy 
with science. Additionally, TPD did not incorporate principal training. 

Inquiry learning and improving science education was the primary focus of the 
California Math and Science Partnership (CaMSP) professional development program 
(Buczynski & Hansen, 2010); inquiry is a component of CSM as is the goal to improve 
science education. CaMSP professional development utilized a similar format as CSM, 
and, like CSM qualitatively measured teacher knowledge changes, and employed control 
groups of teachers and their students to better understand how the professional 
development affected student achievement. Unlike CSM, whose researchers observed 
each participant teacher twice to evaluate change in teacher practice, CaMSP only 
included a small sample of teacher observations. Additionally, CaMSP did not include 
principals in the program while CSM provided principals with training in CSM and 
inquiry strategies, and interviewed principals during the school year to evaluate whether 
CSM had affected the principals’ practices. 

Finally, Berlin and White (2010) offer training in integration strategies and 
content knowledge of science, mathematics and technology, but their program is for 
preservice teachers as part of a Master’s of Education program called Mathematics, 
Science, and Technology Education (MS AT). MS AT is designed for graduate students 
with bachelor’s degrees in science, mathematics or technology who wish to earn teacher 
licensure concurrently with a master’s degree in education. The program is standards- 
based and encourages participants to readily identify content overlaps between 
disciplines. Data gathered addressed whether the program altered participants’ 
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perceptions regarding integration; no measure for change in participants’ content 
knowledge of teaching skills was acquired. Additionally, being a preservice teacher 
program, no student or administrator participation was possible. 

To date, CSM is the first model to incorporate teacher and principal training in 
integration and inquiry, measure results based on teacher, principal, and student 
outcomes, and provide longitudinal data on how the model affects teacher practice and 
student achievement. CSM has also undergone yearly evaluation such that program 
directors have continually improved the program. 

Results from prior CSM cohorts. 

Gloyna (2008) evaluated the effectiveness of CSM professional development 
during its first two years of implementation. Three cohorts of teacher participants took 
part during this time frame. No student data were collected and no principals were 
involved with CSM professional development at this point. Each of the three cohorts 
attended two weeks of summer training and several Saturday follow-up sessions during 
the school year. Teacher participants took pretests and posttests to assess change in 
content knowledge, completed program evaluations and were observed during the school 
year by CSM directors. Teacher work completed during the program was also analyzed. 
Gloyna found that teachers of all three cohorts performed significantly better on content 
posttests than on pretests, demonstrating that teacher content knowledge was improved. 
Although mathematics scores were generally higher than science scores, improvements in 
science were more significant than improvements in mathematics. Gloyna suggested that 
the score difference implied that teachers’ mathematics background knowledge was 
higher than their science background knowledge. 

Teachers expressed commitment to CSM facets and reported understanding 
correlated lessons better after participation in CSM. However, classroom observation 
data showed that teachers did not teach correlated lessons in their classes although 
aspects of science and mathematics integration were observed. Teachers reported several 
obstacles to implementation of correlated and integrated lessons. Teachers were not 
comfortable teaching outside of their primary discipline, felt pressure to teach TAKS 
objectives, and did not have support from their administrations to try correlated or 
integrated lessons. Additionally, teachers lacked materials and both personal planning 
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time and joint planning time with their science or mathematics partner teacher (Gloyna, 
2008) 

Duran (2010) evaluated the third year of CSM implementation - the fourth cohort 
of participant teachers. Teachers again attended two weeks of summer training and 
several Saturday training sessions during the school year. Their content knowledge was 
measured via content pretests and posttests and the degree of teacher change in practice 
was assessed via classroom observations, teacher interviews, and principal interviews. 
Students of participant teachers were given content pretests and posttests to measure the 
degree of student learning. Principals were not directly involved in the professional 
development. 

Duran (2010) found that teachers’ content knowledge in science was significantly 
improved; teachers performed significantly better on content posttests than pretests in 
both space science and geology. Duran found no significant difference in teachers’ 
pretest and posttest scores on the mathematics content test and suggested that too little 
time was spent on mathematics during CSM as both classes were science-led. Teachers 
expressed increased confidence in topics covered during CSM such as the rock cycle and 
phases of the moon. 

Participant teachers gained experience with inquiry and using manipulatives at 
CSM and teachers increased their use of manipulatives during the AY (Duran, 2010). 
Teachers expressed interest in integrating lessons, but reported obstacles that prevented 
them from doing so. Similar to Gloyna (2008), Duran reported that teachers lacked 
planning time and were not able to meet with their correlative partners, received pressure 
to stick to TAKs objectives, and did not feel confident enough to teach two subjects well. 

Based on analysis of descriptive statistics, Duran (2010) detennined that students 
of participant teachers performed better on the mathematics and science posttests than 
they did on the pretests. No significance tests were completed however, therefore it is 
unknown if the students performed significantly better on the mathematics and science 
posttests than on the pretests. 

Research Hypothesis 

The evaluation of the 2009-2010 CSM professional development program will 
address how the program has affected participant teachers, their students, and participant 
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principals. I hypothesize that teachers, principals, and students will be positively 
affected. Specifically, I hypothesize that (a) teacher posttest scores will be significantly 
higher than their pretest scores on both the physics and mathematics content tests; (b) 
qualitative data will provide examples of teachers incorporating integration and inquiry 
teaching strategies into their classroom practices; (c) principals and teachers will express 
positive perceptions of CSM; (d) principals will demonstrate increased support of 
participant science and mathematics teachers trying new integration strategies and will 
demonstrate increased understanding of what effective science and mathematics teaching 
looks like; (e) the percentage of students who met science and mathematics TAKS 
standards in 2010 will be higher for participant teachers than for non-participant teachers; 
and (f) students of participant teachers will demonstrate a significantly larger degree of 
achievement than students of control teachers as measured by science and mathematics 
pretests and posttests. 
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Chapter III 
Methodology 

The purpose of this thesis was to evaluate the effectiveness of the 2009-2010 
CSM professional development program during its fourth year of field testing. Guskey’s 
(2000) five-level evaluation model served as the framework for this external evaluation. 
The evaluation used archival data collected by the CSM researchers during the program’s 
implementation that had not yet been analyzed or interpreted. A mixed-methods research 
model was used to assess how well the CSM program goals were met, how well the 
2009-2010 CSM professional development met accepted standards for effective 
professional development, and how participant teachers and principals and their students 
were affected. 

Research Design 

Mixed-methods research is commonplace and can be a strong research design 
because multiple sources of evidence are built into the design (Johnson & Christensen, 
2008). Research questions can be addressed from a variety of quantitative and qualitative 
angles. The strengths of this evaluation lie in such triangulation (Johnson & Christensen, 
2008; Merriam, 2002). For example, the level of student achievement was assessed via 
two constructs and included analysis of control group achievement; the degree of change 
in participant teacher content knowledge was measured both quantitatively through 
comparison of pretest and posttest scores and qualitatively through analysis of teacher 
reflections and classroom observations; the extent to which CSM strategies are 
incorporated into classroom practice was assessed via analysis of classroom observations, 
interviews and reflections; and the amount of change in principal management styles 
towards science and mathematics teachers was evaluated via analysis of principal 
interviews and reflections. This study also builds on two previous evaluations of four 
prior CSM field test cohorts (Duran, 2010; Gloyna, 2008) and therefore previously 
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documented analyses and themes were revisited for corroboration, convergence, and 
elaboration (Johnson & Christensen, 2008). 

Procedure 

The CSM professional development has been in field testing since 2005. Each 
year has built upon the lessons learned from prior years and has incorporated an 
expanded scope. The 2009-2010 CSM Professional Development Program offered 
sessions for teacher and principal participants and CSM researchers planned for the 
collection of an array of quantitative and qualitative data from participant and control 
teachers and their students, and participant principals. The 2009-2010 CSM professional 
development offered concurrent courses in Correlated Physics and Mathematics (CPM), 
Correlated Mathematical Reasoning and Science (CMRS) and Project-Based Learning 
(PBL) for participant teachers and separate sessions on CSM for principals. 

Participants were requested by CSM researchers to provide demographic 
information, transcripts, and certification records. Teacher participants were invited to 
participate as a team of one science teacher and one mathematics teacher. Teachers 
attended a two-week summer institute in July, 2009 for 75 hours of professional 
development. On the first day of the summer session teachers completed pretests for the 
CPM and CMRS courses and a survey on participants’ satisfaction with integrated 
science and mathematics lessons. Daily, teachers participated in three course lessons per 
day and after every lesson completed a lesson reflection which CSM instructors used as 
formative assessment to alter the next day’s agenda and teaching strategies to best suit the 
needs of the participant teachers. At the end of the summer institute, teachers completed 
posttests for the CPM and CMRS courses. 

CSM researchers planned for teacher courses to incorporate content, content 
pedagogy, inquiry, active learning, cooperative learning, and time for teachers to develop 
their own lessons - components of effective professional development (Birman et ah, 
2000; Guskey & Yoon, 2009; Loucks-Horsley et ah, 2010). During instruction and 
facilitated discussion, course instructors stressed the seven facets of the CSM model - 
teaching for conceptual understanding, using discipline-proper language, identifying 
natural links between disciplines, identifying parallel ideas between disciplines, 
recognizing language students find confusing, using standards-based objectives, and 
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incorporating 5E and inquiry (West & Browning, 2010). CPM content focused on force 
and motion, CMRS content focused on fractions and proportions, and PBL content 
focused on energy transformation via group design projects. 

Participant and control teachers were asked to provide TAKS data for their 2008- 
2009 and 2009-2010 students at the beginning and end of the Academic Year (AY), 
respectively. CSM researchers provided both sets of teachers with student pretests and 
posttests in either science or mathematics to administer to their students at the beginning 
and end of the AY. 

Teacher participants attended four Saturday sessions throughout the 2009-2010 
AY and were offered stipends to attend either the Conference for the Advancement of 
Science Teaching (CAST) or the Conference for the Advancement of Mathematics 
Teaching (CAMT). Saturday sessions offered further training in correlated content and 
content pedagogy as well as an opportunity to discuss implementation of and feedback on 
correlated lessons. The last Saturday session in May, 2010 was allotted for teacher 
science-mathematics teams to work together on correlated lessons for the next school 
year, receive immediate feedback from instructors and peers, and share ideas. Teachers 
were interviewed at the May Saturday session for year-end impressions of the 2009-2010 
CSM program. 

Principal participants attended two days of principal training during the summer 
2009 institute where they were introduced to the CSM model, and led through 
discussions of science and mathematics standards requirements, best practices, resource 
acquisition, assessing CSM lessons, and promoting effective teaching strategies to their 
staff. Principals were also given the opportunity to experience an inquiry lesson and 
observe correlated lessons. Each day, principals were administered a pretest in the 
morning, a posttest in the afternoon, and completed a reflection of the day’s activities. At 
the end of the second day, principals also completed an evaluation of the summer CSM 
training module. 

Principals attended the same four AY training dates as the teacher participants, 
but participated in separate training sessions. At these AY trainings, principals worked 
on identifying and developing solutions for barriers principals face in initiating and 
maintaining collaborative, integrative science and mathematics programs, and 
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understanding the roles principals play in correlated instruction and evaluation. 

Principals also had opportunities to share challenges and best practices, and observe and 
participate in the teacher CSM training. 

In addition to the training sessions, teachers were observed and interviewed twice 
during the AY by the CSM researchers, once in the fall and once in the spring. Teachers 
were asked to teach a correlated lesson on observation days, when possible. Principals 
were interviewed by the primary researchers during the AY when the researchers were on 
each principal’s campus perfonning teacher interviews. 

The evaluation of the 2009-2010 CSM program’s effectiveness used the archival 
data collected throughout the program implementation to determine whether the program 
met its goals, aligned with standards and characteristics of effective professional 
development, and positively impacted participant teachers and principals and their 
students. Detennination of participants’ reactions to the professional development 
program constitute level one of Guskey’s (2000) professional development evaluation 
model. Qualitative data from participant and teacher daily reflections and interviews 
were analyzed for participants’ perceptions of the CSM program in general and 
indications of the degree of satisfaction with the program’s format, amount of active 
learning, and relevance of content. 

Level two of Guskey’s (2000) professional development evaluation model calls 
for an assessment of the participants’ learning. Teacher scores from the CPM and CMRS 
pretests and posttests tests were used to measure the degree to which teachers increased 
their knowledge of physics and mathematics CSM strategies as a result of CSM 
participation. Results of the test score analyses were combined with qualitative analysis 
of participant teacher reflections and interviews to develop a broader characterization of 
the nature of participants’ understanding of content and CSM strategies. Data from 
administrator pretests and posttests was not available for analysis. Although the degree 
of principal knowledge increase was not able to be detennined, qualitative analysis of 
principal reflections indicated the nature of what principals learned during CSM. 

Level three (Guskey, 2000) analysis determined the degree to which the CSM 
program garners support from the participant school administrations and facilitates 
change in those administrations or school culture. The level of collective participation 
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was assessed from the participant demographic and attendance records. Teacher and 
principal reflections and interviews as well as teacher observations were analyzed for 
evidence of teacher collaboration, principal support and encouragement of participant 
teachers, and coherency of the CSM program with participant schools’ goals and 
contexts. 

Assessment of whether and to what degree CSM participants applied knowledge 
and skills acquired during the professional development is called for by level four of 
Guskey’s (2000) evaluation model. This assessment was made for teacher participants 
via qualitative analysis of teacher observations, interviews, and to a lesser degree, 
summer institute reflections. The analysis looked for evidence of teachers using 
integrated or correlated lessons, using inquiry lessons, making links between science and 
mathematics explicit for students, and clarifying for students the similarities and 
differences in language use between science and mathematics. Evaluation of how 
participant principals applied CSM skills came primarily from qualitative analysis of 
principal interviews, but also from teacher interviews and observations. The analysis 
looked for evidence that principals altered the way they evaluated science and 
mathematics teachers during classroom walk-throughs, encouraged or otherwise support 
participant teachers’ use of CSM teaching strategies, and facilitated teacher collaboration. 

Evaluation of CSM at Guskey’s (2000) level five involves assessing student 
learning. Three data sources were analyzed to evaluate to what degree teacher 
participants in CSM positively impacted their students’ learning. The students’ pretest 
and posttest scores on science and mathematics content tests were compared for 
significant achievement differences between students of participant teachers and students 
of non-participant teachers. In order to estimate whether teacher participation in CSM 
relates to student achievement on the TAKS examinations, the percentage of students 
who met TAKS standards and the percentage of students who earned commended scores 
in 2009 was compared to the 2010 percentages. Comparisons were made for those 
teachers who submitted both years’ score sets who also taught the same grade and 
discipline in 2009 and 2010. Additionally, teacher observations and teacher and principal 
interviews were analyzed for anecdotal evidence of student learning. 
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Participants 

The CSM researchers used quota and convenience sampling to obtain participant 
school districts and schools for the 2009-2010 cohort five of CSM. The grant funding the 
CSM project required that at least one participating school district was high needs, 
meaning at least 20% of the student body qualified for free or reduced lunch which in 
Texas is a proxy for quantifying economic disadvantage (TEA, n.d.). Four school 
districts participated in the 2009-2010 program; these districts were located in rural and 
urban areas of central Texas and served a range of 500 to 70,000 students each. All four 
districts have a large proportion of students on free or reduced lunch; between 40% and 
60% of students at three districts are economically disadvantaged, and nearly 98% of 
students at the fourth district are economically disadvantaged (TEA, n.d.). 

Six schools from the four school districts participated in the 2009-2010 CSM. 

The principals of each school recruited science and mathematics teachers of the fifth 
through eighth grades to participate in teams or serve as controls, and seven principals 
from the six schools also participated themselves to varying degrees. Six of these 
principals were female, one was male. Five of the seven principals were white and two 
were African American. All held Master’s degrees. Principals received stipends for 
participation, pro-rated per attendance. 

Eighteen science and mathematics teachers of grades five through eight 
participated in the 2009-2010 CSM program. The aim of CSM was for teachers to attend 
with a colleague of their complementary discipline such that they could work together 
through the school year. The majority of participating teachers did attend in science- 
mathematics teams of two to three teachers. Two teachers participated without a 
complement discipline partner. Teacher participants included four 5 th grade teachers, of 
which one taught science, one taught mathematics, and two taught both science and 
mathematics; seven 6 th grade teachers, of which three taught science and four taught 
mathematics; one 7 th grade teacher who taught mathematics; four 8 th grade teachers, of 
which one taught science, two taught mathematics, and one taught both science and 
mathematics; and one 7 th and 8 th grade teacher who taught science. Thirteen teacher 
participants were female and four were male. Eleven teachers self-identified as White, 
three self-identified as Hispanic, one self-identified as American Indian or Alaskan 
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Native, and two self-identified as other. No teachers self-identified as African American 
or Asian/Pacific Islander. All teachers held Bachelor’s degrees, and additionally, four 
teachers held Master’s degrees in education, one held an MBA in science and technology, 
and one held both a Master’s degree in microbiology and an MBA in business. Fifteen 
teachers were certified to teach in Texas at the beginning of the 2009-2010 CSM 
professional development program and two teachers were not certified to teach in Texas; 
five of the certified teachers had obtained their certifications alternatively. Participant 
teachers also received stipends for participation, pro-rated per attendance. 

Four of the six participating schools recruited eleven sixth through eighth grade 
science and mathematics teachers to serve as control groups. Control teachers included 
four 6 th grade teachers, of which one taught science, one taught mathematics, and two 
taught both science and mathematics, two 7 th grade teachers, of which one taught 
mathematics and one taught both science and mathematics, three 8 th grade mathematics 
teachers, and two 7 th and 8 th grade teachers, of which one taught science, and one taught 
mathematics. Education history and teacher certification status was not collected for 
control teachers. These teachers did not participate in CSM training and were neither 
observed nor interviewed, but did administer the science and mathematics pretests and 
posttests to their students and were asked to provide student TAKS data to CSM 
researchers. Control teachers did not receive stipends. 

Students of both participant and control teachers were administered pretests and 
posttests in either science or mathematics depending on teacher discipline in order to 
determine if teacher participation in the 2009-2010 cohort of CSM indirectly impacted 
student achievement. As shown in Table 2, students in the participating districts were 
predominantly Hispanic and economically disadvantaged (TEA, n.d.). Students at each 
school were randomly assigned to classes at the beginning of the school year. Classes 
were heterogeneous and inclusive - students were not ability- tracked. 
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Table 2 

School District Student Demographics 



District 


Grades 

Serves 


% African 
American 


% Hispanic 


% White 


% Native 
American 


% Asian / 
Pacific 
Islander 


% 

Economically 

Disadvantaged 


Total No. 
Students 


A 


K-12 


1.1 


77.5 


20.7 


0.2 


0.4 


72.1 


1711 


B 


PreK-12 


9.8 


85.1 


4.8 


0.0 


0.2 


99.2 


498 


C 


K-12 


9.4 


49.4 


36.8 


0.3 


4.1 


42.8 


65217 


D 


K-12 


0.8 


59.4 


39.4 


0.3 


0.1 


61.2 


1913 


Weighted 
Average, % 




8.9 


50.6 


36.2 


0.3 


3.9 


44.5 





Note. Data is for the 2009-2010 school year and was obtained from The Texas Education Agency’s Academic Excellence Indicator System accessible from 
http://ritter.tea.state.tx.us/perfreport/aeis/ 
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Teacher attendance at CSM training sessions. 

Teachers attended CSM professional development sessions at high rates. 

Teachers were asked to attend a two-week summer institute, four AY Saturday trainings, 
and were given a cash incentive to attend CAST. Teacher attendance details are shown in 
Table 3 below. Attendance at the summer institute was very high. Sixteen of the 18 



Table 3 

Teacher Attendance at CSM Training Sessions 



Teacher 


Summer Institute 




AY Saturday Sessions 




No. Days 
Attended 


Sept 


Oct 


CAST 


Jan 


May 


A 


10 


Y 


Y 


N 


Y 


Y 


B 


10 


Y 


Y 


N 


Y 


N 


C 


5 


Y 


Y 


Y 


N 


Y 


D 


10 


Y 


Y 


N 


N 


Y 


E 


10 


Y 


Y 


Y 


N 


Y 


F 


9 


N 


Y 


N 


N 


N 


G 


10 


Y 


N 


N 


Y 


Y 


H 


10 


Y 


N 


Y 


N 


Y 


I 


8 


Y 


Y 


Y 


N 


Y 


J 


9.5 


Y 


N 


N 


Y 


N 


K 


10 


Y 


Y 


N 


Y 


Y 


L 


5 


Y 


N 


N 


Y 


Y 


M 


9 


N 


N 


Y 


Y 


N 


N 


8 


Y 


N 


N 


Y 


Y 


0 


10 


Y 


N 


N 


Y 


Y 


P 


10 


Y 


Y 


Y 


Y 


Y 


Q 


10 


Y 


Y 


Y 


Y 


Y 


R 


10 


N 


Y 


N 


N 


Y 



Notes. Y = attended; N = did not attend; Total number of days possible to attend the summer institute was 



10 . 
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(89%) participant teachers attended at least eight of the ten days of summer training. 
Eleven participated in all ten days, five attended at least eight days, and two teachers 
attended only five days of training. During the AY, attendance was highest at the 
September and May sessions and slumped during the middle of the AY. Two teachers 
(11%) attended every training session and another two teachers (11%) attended every 
function except CAST. All teachers attended at least 53% of the offered training days. 

Principal attendance at CSM training sessions. 

Principals of each of the campuses participated to a lesser degree than did the 
teachers. Principals were requested to attend two days of training during the summer 
institute and four Saturday sessions throughout the school year. Also, just as the teachers 
were, principals were offered a cash incentive to attend CAST, but no principal did. 
Principal attendance detail is shown in Table 4 below. Discounting CAST attendance, 



Table 4 

Principal Attendance at CSM Training Sessions 



Principal 


Summer Institute 




AY Saturday Sessions 




Day 1 


Day 2 


Sept 


Oct 


CAST 


Jan 


May 


T 


Y 


Y 


Y 


Y 


N 


Y 


Y 


U 


N 


Y 


N 


N 


N 


N 


N 


V 


N 


N 


N 


N 


N 


Y 


N 


W 


Y 


Y 


N 


N 


N 


N 


N 


X 


Y 


Y 


Y 


Y 


N 


N 


N 


Y 


N 


N 


N 


N 


N 


N 


N 


Z 


Y 


Y 


N 


N 


N 


N 


N 


Notes. Y = 


attended; N 


= did not attend. 













principal attendance ranged from perfect attendance at all trainings to no attendance at 
any training. Four principals (57%) attended both days of the summer institute, one 
(14%) attended one day, and two (29%) attended neither day. One principal (14%) 
attended all four Saturday training sessions, one (14%) attended two Saturdays, one 
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(14%) attended only one Saturday, and four principals (57%) did not attend any Saturday 
trainings. 

Instrumentation 

CSM researchers planned for the collection of a variety of data during 
implementation of the 2009-2010 CSM professional development program. Both 
quantitative and qualitative instruments were developed by the researchers and employed 
to understand how the CSM professional development affected participants’ knowledge 
and skills. All instruments were created by the CSM researchers or summer institute 
instructors. 

Quantitative instruments. 

Several quantitative instruments were developed by CSM researchers and 
instructors to measure how the CSM professional development program impacted 
participant teachers’ content knowledge and indirectly affected the academic 
performance of involved students. Quantitative instruments included teacher pretests and 
posttests for the courses CPM and CMRS, and student pretests and posttests in science 
and mathematics. Additionally, researchers requested student TAKS performance data 
from participant teachers for two academic years. 

Teacher tests. 

CSM researchers administered content pretests in physics and mathematical 
reasoning to participant teachers at the beginning of the summer institute and 
administered the same two content exams as posttests at the end of the summer institute. 
The physics test consisted of 15 questions, nine of which were multiple choice and six of 
which were open-ended short answer. The topics covered included: force, motion, 
distance, displacement, scientific method, graph construction and interpretation, and 
experimental variables. Questions for the physics test were developed by the physics 
professional development instructor, an Assistant Professor of Physics at a large southern 
state university. The reliability and validity of this physics test has not been determined. 
The teacher pretest and posttest on quantitative reasoning contained 25 questions of 
which 17 were multiple choice and eight were open-ended short answer. Tested content 
included: number properties, order of operations, proportions, fractions, word problems, 
equation development, unit conversions, interpretation of charts and graphs, and 
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mathematical reasoning. Test questions were taken from multiple sources. Seven 
questions are from a 2009 mid-term examination given to undergraduate education 
students enrolled in Mathematics Methods for EC-6 at a small southern university. The 
reliability and validity of these questions have not been detennined. Nine questions are 
from the 2008 released Mathematics Texas Assessment of Knowledge and Skill (TAKS) 
for eighth grade. Reliability and validity of TAKS tests will be discussed in the section 
on Student TAKS data. Remaining questions were taken from various publications by 
the National Council of Teachers of Mathematics, National Assessment of Educational 
Progress (NAEP), and an analysis of and suggestions for assessment of mathematics 
teachers’ content and pedagogical content knowledge by Hill, Sleep, Lewis and Ball 
(2007). 

Student tests. 

Students of participant and control teachers were administered pretests and 
posttests in physics or mathematics, depending on the discipline of the teacher, at the 
beginning and end of the 2009-2010 school year. Pretests and posttests for each 
discipline were the same. The physics test consisted of 19 multiple choice questions 
covering the topics: force, motion, energy, systems, scientific method, and interpretation 
of charts and graphs. Questions were taken from the Spring 2009 released Massachusetts 
Comprehensive Assessment System (MCAS) Grade 5 Science and 
Technology/Engineering Test, the 2008 released Virginia Standards of Learning (SOL) 
Assessment for grade 5 science, the 2001 and 2002 released New York Regents Exams 
(Regents) for grade 8 science, and the 2005 released Colorado Student Assessment 
Program (CSAP) grades 5 and 8 science demonstration questions. The mathematics test 
consisted of 25 multiple choice questions covering the topics: fractions, proportions, 
percentages, statistics, and interpretation of charts and diagrams. Similarly, questions for 
the mathematics test were also pulled from annual state assessments - released MCAS 
mathematics tests for grades 5 through 8, and released Virginia Math assessments for 
grades 5 and 6. 

Annual state assessments are required as part of the No Child Left Behind Act of 
2001 (H. R. 1, 2002) and states strive for valid and reliable tests to meet this requirement. 
All states report a rigorous protocol for developing, reviewing, field testing, and re- 
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reviewing test questions which maintains content validity to the state-specific curriculum 
standards, maintains high degrees of internal consistency, and minimizes bias (Colorado 
Department of Education [CDE], 2007; Massachusetts Department of Education [MDE], 
2007; New York State Education Department [NYSED], 2002; TEA, 2008; Virginia 
Department of Education [VDE], 2005). “Achievement tests are typically considered of 
sound reliability when their reliability coefficients are in the range of 0.80 and above” 
(CDE, 2007, p. 69). 

MCAS reports Cronbach’s alpha ranges of 0.85 - 0.93 and stratified alpha ranges 
of 0.87 - 0.93 for all 2009 assessments (MDE, 2009). Validity is additionally ensured 
several ways. Between 1999 and 2006, Massachusetts was able to criterion reference the 
MCAS to numerous national standardized tests including the Metropolitan Achievement 
Test (MCAT-7), the Stanford Achievement Test (SAT-9), the Iowa Test of Basic Skills 
(ITBS), and the NAEP. Studies completed for internal structure and convergent evidence 
via comparison to classroom performance also suggest high validity (MDE, 2007). 

Virginia Department of Education (2005) reports KR20 reliability of 0.91 for the 
2004 Grade 5 Science exam. The reliability coefficient of other 2004 non-writing 
examinations ranges from 0.85 -0.94 and it is reasonable to assume reliability is 
consistent year after year. 

One reported goal of the New York State Education Department’s Office of State 
Assessments is to develop tests that are comparable in “fairness, validity and reliability” 
(NYSED, 2002, slide 16) across different forms, and one may infer, across different test 
years. Although reliability data on New York’s science tests were not accessible, the 
2009 intermediate grades mathematics test had a Cronbach’s alpha reliability range of 
0.88 - 0.94 (NYSED, 2009) and the 2001 fourth grade mathematics test had a Feldt-Raju 
reliability of 0.94 (NYSED, 2001). Mathematics test reliabilities remain high over time 
and one may infer that similar standards are required for the science tests. 

Recent technical reports for the CSAP document consistently yield high reliability 
coefficients for the science CSAPs. The reliability coefficient for 2007 was 0.92 - 0.94, 
for 2008 was 0.92 - 0.93 and for 2009 was 0.92 - 0.93 (CDE, 2007, 2008, & 2009). 

Older technical reports were not available on the Colorado Department of Education’s 
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website but one may infer from this small sample that older science CSAPs also had high 
reliability coefficients. 

Student TAKS data. 

During the time of the study, Texas students in grades 5 through 8 took a 
Mathematics TAKS examination near the end of each school year and students in grades 
5 and 8 also took a Science TAKS examination. CSM researchers asked participant 
teachers to report the percent of their students who met TAKS standards for the 2008- 
2009 and 2009-2010 school years. 

TAKS is a standards-referenced assessment used to infer students’ level of 
knowledge and understanding of various subjects’ Texas Essential Knowledge and Skills 
(TEKS) (TEA, 2008). TAKS reliability of multiple choice tests, such as the Mathematics 
TAKS and Science TAKS, is calculated via the Kuder Richardson Fonnula 20 (KR 20); 
internal consistency ranges from .87 - .90. The validity of the TAKS is sound, and is 
continually being studied. Content validation is checked at multiple levels for each test; 
TEA asks separate committees of educators, TEA staff, and testing professionals to verify 
the TAKS matches the TEKS it strives to assess. Risk of single-sample bias is minimal 
because multiple authors write and review the questions. TAKS has been acceptably 
criterion-referenced against the Texas Academic Skills Program (TASP), the American 
College Test (ACT), and the Scholastic Assessment Test 1 (SAT I). Additionally TAKS 
pass/fail rates have been satisfactorily compared with classroom pass/fail rates (TEA, 
2008). 

Qualitative instruments. 

Numerous qualitative instruments were developed by CSM researchers and used 
to assess how the CSM professional development program was received by participants, 
and affected participants’ knowledge, practices, and opinions regarding science and 
mathematics integration and correlation. Qualitative instruments included teacher 
surveys, reflections, classroom observations and interviews, and principal pretests and 
posttests, reflections, program evaluations, and interviews. 

Teacher survey and reflections. 

During the summer institute, CSM researchers had participant teachers complete a 
survey of their satisfaction with integrated science and mathematics lessons (see 
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Appendix A). The survey asked teachers how often they taught integrated lessons per 
week and where they got the outlines for such lessons. Teachers who indicated they did 
integrate regularly were asked to indicate whether each of eleven potential barriers to 
integration were major or minor problems. Finally all teachers were asked for their 
opinion on the reasons for and the value of integrating. 

Teacher participants completed reflections three times a day at the summer 
institute, one reflection after each CPM, CMRS, and PBL professional development 
session (see Appendix B). Participants were asked to comment on their most and least 
favorite aspects of the day’s lesson, whether any concepts were new, how their prior 
knowledge was deepened or broadened, and what other ideas or TEKS that day’s lesson 
brought to the participant’s mind. CSM researchers and instructors used the reflections 
as daily formative assessments to hone the next day’s or next summer’s sessions, to best 
meet the program goals and the needs of the teacher participants. 

Teacher observations and interviews. 

Classroom observations of participant teachers were conducted by the two 
primary researchers for the 2009-2010 CSM project. Both researchers have extensive 
science or mathematics K-12 teaching and supervisory experience, as well as experience 
teaching, observing, and mentoring preservice teachers at the university level. Each 
participant teacher was observed and interviewed twice - once during fall semester 2009 
and once during spring semester 2010. The researchers took field notes then transferred 
the combined field notes to a standard CSM teacher observation and interview form (see 
Appendix C) which included a checklist for effective science teaching characteristics 
such as use of enhanced context, collaborative learning, questioning, inquiry, 
manipulatives, testing, instructional technology, enhanced materials, and direct 
instruction for process. Researchers looked for evidence that participant teachers were 
incorporating CSM strategies and the seven CSM facets into their teaching. Researchers 
probed the participants for their classroom teaching goals, current level of CSM 
understanding and usage, barriers they encountered during CSM implementation, and 
ideas they had developed, and offered support to the teachers. Researcher-developed 
interview questions for teachers are included in part II of Appendix C. 
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Teacher May interview. 

Each teacher participant was briefly interviewed at the last Saturday session in 
May to understand the cohort’s assessment of the CSM program after one year of 
participation. Teachers were asked to identity the most and least valuable portions of the 
professional development program, and reflect on how they have altered their teaching 
practices and how their students may have responded to the changes. 

Principal tests. 

Principal participants attended the CSM summer institute for two days; each day 
they took a pretest in the morning and a posttest in the afternoon on CSM and inquiry 
strategies for administrators. The test was different each day, but each day’s pretest was 
identical to the posttest. The first day’s test consisted of 16 open-ended questions and 
ranking exercises and the second day’s test had 14 questions. The tests were developed 
by the session instructor and as such were designed to engage the participant principals 
with the session’s agenda and later assess their new knowledge. Topics included TEKS, 
resource availability and acquisition, teacher observations and support, using effective 
teaching strategies in staff meetings, and supporting CSM within the school. Copies of 
the participants’ test and the test results were not available for analysis. 

Principal reflections and evaluations. 

As with the teacher participants, principal participants completed daily reflections 
(see Appendix D) and an end-of-session evaluation. The reflections asked principals to 
consider how they will use perfonnance data to improve instruction; facilitate integration; 
transfer student engagement strategies to other disciplines; alter their observation process 
to promote integration; facilitate teacher collaboration; and measure whether CSM has 
been effective at their school. The postsession evaluation had principals rate how well 
the principal CSM session’s objectives were met for principals, teachers, and students. 
The principal reflections were available for analysis, but the end-of-session evaluations 
were not. 

Principal interviews. 

Available participant principals were interviewed once during the school year by 
the two primary CSM researchers. Researcher-developed interview questions for 
principals are included in part III of Appendix C. Researchers recorded field notes 
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during the interviews and later combined their notes into a joint summary. Principals 
were asked to discuss their goals for science and mathematics instruction; any changes 
they noticed in teacher instructional practices and the science or mathematics programs; 
how student perfonnance data may reflect teaching practices; how they model or 
otherwise disseminate effective teaching strategies to teachers; resource allocation and 
acquisition; strategies to improving student and teacher attitudes towards science and 
mathematics; and impressions of the CSM professional development program. 

Data Analysis 

The archival data collected via the above instruments during the 2009-2010 CSM 
professional development program were analyzed using mixed methods to assess the 
effectiveness of the CSM program and address the seven proposed research questions. 
Quantitative data such as teacher and student pretest and posttest scores and student 
TAKS data were analyzed via comparative statistics, descriptive statistics and graphical 
analysis. Qualitative data such as surveys, reflections, observations, and interviews were 
interpreted via the constant comparative method of qualitative analysis (Merriam, 2002; 
Strauss & Corbin, 1990) then synthesized with the quantitative results to develop 
conclusions. 

Quantitative analysis. 

Statistical analyses of the participant teachers’ scores on the CPM and CMRS 
pretests and posttests addressed research question one which asks whether participant 
teacher content knowledge significantly improves as a result of participation in the CSM 
summer institute. The data for the CPM and CMRS tests were separately analyzed via 
related-samples t-test (Gravetter & Wallnau, 2007; Green & Salkind, 2005). A related- 
samples t-test assumes that (a) population difference scores are distributed normally, and 
(b) each case is a random sample from the population and pretest and posttest scores are 
independent (Gravetter & Wallnau, 2007; Green & Salkind, 2005). Both assumptions are 
met by both the CPM and CMRS data sets. An alpha level of 0.05 was used to judge 
significance of the mean difference between pretest and posttest scores. Effect size was 
calculated with both Cohen’s d and r, the percentage of variance accounted for 
(Gravetter & Wallnau, 2007). 
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Although two principal pretests and poshests were administered to principal 
participants during the summer institute, the data from these tests was not available for 
analysis. No results are reported or discussed for principal pretests and posttests. 

Research question six asks how student performance on TAKS compares first 
between participant and non-participant teachers and second between participant teachers 
before and after attending CSM professional development. No analysis was completed 
comparing the TAKS scores of participant and non-participant teachers because no non- 
participant teacher TAKS scores were collected. Separate binomial tests were performed 
for each participant teacher that submitted two years of TAKS data to determine whether 
the perfonnance of their 2010 students was significantly higher or lower than the 
performance of their 2009 students. For each teacher, a separate binomial test was run to 
compare the percentage of students who met TAKS standards between 2009 and 2010 
and the percentage of students who met TAKS commendation standards between 2009 
and 2010. The 2009 student performance metric was used as the hypothesized value each 
2010 metric was compared against. Alpha levels of 0.05 were used to judge whether 
results were significant. For significant results, effect size was calculated from P 0 b se rved - 
Phypothesized (Green & Salkind, 2005, p 352) 

Research question seven which asks whether the CSM professional development 
had an indirect influence on student learning of science and mathematics content was 
analyzed with two separate one-way analyses of covariance (ANCOVA) models. The 
models analyzed differences between the perfonnance of students of participant and 
control science and mathematics teachers on the content posttest controlling for the 
students’ performance on the pretest. An analysis must meet four assumptions for 
ANCOVA results to be valid. First, the poshest scores must be normally distributed in 
the population, and there was no reason to assume this to be untrue. Second, the 
variances of the posttest scores should be equal and the scores ranges and variances were 
reasonably similar. Third, the posttest scores should be independent from each other, and 
the posttest scores were. Fourth, the pretest scores should be linearly related to the 
posttest scores and the slopes relating the pretest scores to the posttest scores should be 
equal (Green & Salkind, 2005). Assumption four, also known as the homogeneity-of- 
slopes assumption, was tested via a function built into SPSS (Version 19) and was passed 
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for both the science and mathematics content test data sets. Alpha levels of 0.05 were 
used to measure significance. Effect sizes were calculated with partial rf (Green & 
Salkind, 2005). 

Additionally, I was interested in determining whether students performed better 
on each content test if they had more teachers who had participated in CSM. Students 
could have 0, 1, or 2 participant teachers for science and mathematics class in 2009-2010. 
ANCOVA models were attempted for each content test comparing the posttest scores of 
students with 0, 1, or 2 participant teachers with the pretest score held as a covariate. 

Both the science content test model and the mathematics content test model failed the 
SPSS test for the homogeneity-of-slopes assumption however, and therefore the 
ANCOVA models could not be analyzed. 

Qualitative analysis. 

Research questions two through five were examined via qualitative data sources 
and analyzed using the constant comparative method of qualitative analysis (Merriam, 
2002; Strauss & Corbin, 1990). The purpose of the qualitative analysis was to develop 
themes which characterize and explain how participant teachers and principals perceived 
CSM, what participants learned and had difficulty with during the classes, the extent to 
which teacher participants incorporated CSM strategies into their classroom practices, 
and how CSM training affected principals’ management of science and mathematics 
teachers (Bogdan & Biklan, 2007; Merriam, 2002). 

The programs NVivo8 (2008) and later NVivo9 (2011) were used as a tool to 
manage and interpret data from teacher and principal interviews, observation field notes 
and formal checklists, and participant surveys, reflections and evaluations. Data were 
analyzed through several iterations of coding, writing analytic memos, categorizing, and 
modeling (Bazeley, 2007; Saldana 2009; Strauss & Corbin, 2000). During the first cycle 
of analysis, open coding, data were broken down into its component parts, considered on 
its own merit as well as compared with previously read data, and coded according to a 
preliminary scheme (Strauss & Corbin, 2000). Initial provisional codes were 
brainstonned prior to starting the analysis and additional codes were generated from 
concepts within the data (Saldana, 2009). Codes of several types were used such as: 
descriptive, context, process, emotion, values, attributes, and structural, among others 
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(Bogdan & Bilkan, 2007; Saldana, 2009). As analyses continued, codes were revised, 
and data were often re-coded to reflect newly read data and interpretation ideas (Bazeley, 
2007; Miles & Huberman, 1994; Saldana, 2009; Strauss & Corbin, 2000). During the 
second cycle, axial coding, codes were sorted into hierarchical trees that better reflected 
the characterization frameworks needed to address the research questions, and further 
revised (Saldana, 2009; Strauss & Corbin, 2000). Throughout the analysis, memos were 
written and models of teacher cases were drawn to capture ideas about the data, help 
identify patterns and themes, and reflect on how aspects of the analysis linked to 
literature (Bazeley, 2007; Saldana, 2009; Strauss & Corbin, 2000). Cross-case analysis 
(Bazeley, 2007) of teacher case models helped determine themes in CSM perceptions and 
participant learning. Matrix-style pattern analysis (Bazeley, 2007) was very helpful to 
sort out correlations between participants, events, emotions, and outcomes, among other 
things. The emergent categories, themes, and characterizations were verified and cross- 
checked during the last stage of analysis (Miles & Huberman, 1994; Strauss & Corbin, 
2000). 

Project Validity 

Consideration of a project’s validity is important because it signals to consumers 
of the research what the project’s limitations may be and how readily the results may be 
generalized to other populations or contexts (Johnson & Christensen, 2008). Potential 
internal and external validity issues related to the implementation and evaluation of the 
2009-2010 CSM professional development program are discussed below. 

Internal validity. 

As with all research, internal validity threats exist. Quantitatively, there is a 
testing threat that participants’ posttest scores increased because they took the same test 
as the pretest. Participants are more familiar with the questions during the posttest, and if 
the answers to the pretest were reviewed, which may be possible in the case of the 
student participants, the answers may be recalled for the posttest. Regression artifacts are 
possible on the teacher participant pretests and posttests. Regression is the tendency for 
extreme scores to gravitate towards the mean regardless of treatment; the combination of 
a relatively small number of participant teachers with extremes in prior knowledge 
suggests regression may come into play (Johnson & Christensen, 2008). 
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Qualitatively, because the 2009-2010 CSM professional development is not being 
evaluated by the primary researchers, there is the threat that some critical elements of the 
human instrument (researcher-as-data-collector) will be missed. However, the primary 
researchers are in communication with the evaluator and also peer reviewed the 
evaluation. The researchers are also experienced qualitative data collectors and have 
been continuously involved with CSM since its inception which increases the study’s 
qualitative reliability (Merriam, 2002). 

External validity. 

The CSM professional development model may be limited in its generalizability. 
Participant districts and teachers were not randomly chosen and therefore may not serve 
as a representative sample of national districts and teachers. The weighted average 
student demographics show 50.6% of the students in the districts studied are Hispanic, 
36.2% are White, 8.9% are African American, and 4.2% are other ethnicities. Participant 
teachers were 65% White, 17.5% Hispanic, and 17.5% other ethnicities; no African 
American teachers participated in the 2009-2010 CSM cohort. Incorporation of CSM 
strategies into classroom practices may play out differently in districts with different 
demographic proportions; the effectiveness of CSM strategies in predominantly African 
American populated districts especially may not transfer since so few African American 
students were included in this study and no participant teachers were African American. 

Additionally Johnson (2009) has shown that motivated refonn-oriented teachers 
absorb and internalize effectively-presented teaching strategies at a greater rate than more 
traditional status-quo teachers. Affective data on teachers was not purposefully collected 
on participant teachers during CSM 2009-2010 cohort therefore it will not be known 
whether CSM strategy assimilation follows a similar pattern or is more applicable to 
reform-oriented teachers than non-reform oriented teachers. 
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Chapter IV 
Results 

Analyses of the data collected during the 2009-2010 implementation of the CSM 
program will be presented within the framework of the seven research questions. The 
first three research questions asked how participant teachers perceived CSM, what they 
learned from CSM, and whether they applied new knowledge to their classrooms. 

CSM’s effect on principals - their perceptions of the program and whether they applied 
new knowledge to their management practices - was the focus of research questions four 
and five. The last two research questions queried how CSM indirectly affected students’ 
performance in science and mathematics. 

CSM Effect on Teachers 

Teachers were the primary participants in CSM. Assessment of the effectiveness 
of the CSM program depends largely on how the participant teachers perceived and 
received the professional development, what they learned during the program, and how 
their teaching practices or attitudes changed as a result of having attended. Teachers 
provide important insight into whether and how participant principals altered 
management habits, and the teachers’ ability to apply what they have learned from CSM 
affects their student learning as well. 

Participant teachers’ level of content background knowledge. 

Participant teachers submitted copies of their college transcripts. Submitted 
transcripts were analyzed to understand the college-level content training participants had 
achieved prior to participating in the CSM program. The transcripts showed that only 
22% of participants held an undergraduate degree in science or mathematics and those 
with science degrees were all in biology and biology-related fields (see Table 5). Six of 
the 14 undergraduate degrees categorized as other were in education. 
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Table 5 

Type and Discipline of Degrees Held by Participant Teachers 





BS/BA 

Major 


BS/BA 

Minor 


MS/MA 

Major 


MS/MA 

Minor 


MBA 


Mathematics 


1 


4 


0 


0 


0 


Science 


3 (biology- 
related) 


1 

(chemistry) 


1 (micro- 
biology) 


1 

(chemistry) 


1 (science 
technology) 


Other 


14 


4 


2 


0 


1 



Table 6 shows the level of background science and mathematics content 
knowledge participant teachers had as measured by earned science and mathematics 
college credits. Credits were included in the count if the participant received a passing 
grade of A through D or P (passing) for the course. All of the participant teachers have 
taken at least one science class with the median number of eight science credits or 
approximately two science courses. Several participants who received advanced degrees 
in a biology field skew the total science mean upward to 22.61 credits. Since one of the 
2009-2010 CSM courses is Correlated Physics and Mathematics, it is important to note 
that even though several teacher participants have earned numerous science college 
credits, only 4 of 18 participants have had any college physics courses and the maximum 
number of physics credits earned was eight (two courses). Overall, participant teachers 
earned fewer mathematics college credits than science credits. Both the median and the 
mode were six mathematics credits (approximately two classes). Three teachers did not 
take any mathematics classes in college and yet these three were all teaching 
mathematics. Mathematics classes listed on transcripts as “for educators” or “for 
teachers” were not included in the credit count of Total Mathematics. Three participant 
teachers earned such mathematics for teachers credits. When these courses are added to 
the dataset, all descriptive statistics remain the same except for the mean which increases 
from 9.83 to 1 1.33 mathematics credits. 
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Table 6 



Number of Mathematics and Science College Credits Earned by Participant Teachers 



Participant 

Code 


Discipline 

Taught 


Grade Taught 


Total 

Mathematics 3 


Total Science 


Biology b 


Chemistry 0 


Earth & Space 
Science 


Physics 


A 


M 


6 


3 


5 


0 


0 


0 


5 


B 


S 


6 


3 


4 


4 


0 


0 


0 


C 


M 


6 


44 


10 


7 


3 


0 


0 


D 


M 


7 


0 


19 


16 


3 


0 


0 


E 


S 


6 


3 


68 


56 


12 


0 


0 


F 


M 


6 


6 


6 


6 


0 


0 


0 


G 


M 


5 


6 


8 


8 


0 


0 


0 


H 


M 


6 


39 


31 


19 


12 


0 


0 


I 


B 


8 


6 


3 


3 


0 


0 


0 


J 


B 


5 


0 


7 


4 


0 


3 


0 


K 


S 


6 


3 


8 


5 


0 


3 


0 


L 


S 


5 


3 


6 


0 


0 


6 


0 


M 


s 


7&8 


22 


7 


4 


0 


3 


0 


N 


M 


6 


0 


15 


5 


0 


10 


0 


0 


M 


8 


18 


13 


7 


0 


0 


3 


P 


S 


8 


6 


86 


47 


31 


0 


8 


Q 


B 


5 


6 


8 


4 


0 


0 


0 


R 


M 


8 


9 


103 


57 


38 


0 


8 






Mean 


9.83 


22.61 


14.00 


5.50 


1.39 


1.33 






Mode 


6 


8 


4 


0 


0 


0 






Median 


6 


8 


5.5 


0 


0 


0 






Min 


0 


3 


0 


0 


0 


0 






Max 


44 


103 


57 


38 


10 


8 



Notes. B = both mathematics and science; M = mathematics; S = science. 

a Total Mathematics category does not include mathematics-for-teachers or business statistics courses. 
b Biology category includes anatomy, ecology, forestry, bacteriology and other organism-related courses. 
^Chemistry category includes biochemistry courses 
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For analysis purposes, participant teachers were split into low and high content 
background knowledge groups for science and mathematics. For both science and 
mathematics, the division between low and high content background knowledge groups 
was based on the median number of credits and approximate correlative number of 
classes. Teachers were considered to have low science background knowledge (LSBK) if 
they had completed two or fewer college science classes (less than ten science credits) 
and high science background knowledge (HSBK) if they had completed three or more 
college science classes (ten or more science credits). Teachers were categorized as 
having low mathematics background knowledge (LMBK) if they had completed two or 
fewer college mathematics classes (six or fewer mathematics credits) and high 
mathematics background knowledge (HMBK) if they had completed three or more 
college mathematics classes (more than six mathematics credits). Table 7 depicts the 
number and percentage of participant teachers who fell in each content background 
knowledge category. 



Table 7 

Background Content Knowledge Classification of Participant Teachers 



Background 

Content 

Knowledge 


Science 




Mathematics 


Number of 
Teachers 


Percent of 
Teachers 


Number of 
Teachers 


Percent of 
Teachers 


Low 


10 


56 


13 


72 


High 


8 


44 


5 


28 



Certainly, the majority of participant teachers gained additional mathematics and 
science content knowledge after graduating college from self-learning, other professional 
development trainings and continuing education, among other experiences. There was no 
effort made to gather detailed information on participants’ level of postcollegiate 
learning. However, participants’ current state of knowledge in mathematical reasoning 
and physics was assessed with pretests in those two content areas. 
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Participant teacher perceptions of the CSM program and strategies. 

CSM teacher participants were administered several qualitative instruments 
throughout the summer institute and on Saturday sessions. Qualitative analyses of the 
daily lesson reflections and May interviews were performed to understand teacher 
perceptions of and reactions to the CSM professional development program as a whole 
and to specific situations experienced during the program. See Appendix E for a small 
sample of teacher reflections. Summaries of the resulting teacher perceptions are divided 
into teacher reactions to the teaching strategies employed by the CSM instructors, teacher 
perceptions of each class as a whole, and emotional responses teachers expressed about 
specific situations throughout the summer institute. 

Perceptions of teaching strategies employed. 

The summer institute presented three daily classes in CMRS, CPM, and PBL. 

The instructors employed a variety of teaching strategies during the lessons. The 
teaching strategies participant teachers most enjoyed experiencing from a student 
perspective and were most excited about adding to their own repertoires were dominantly 
student-centered and include teaching with math manipulatives, hands-on technological 
aspects, collaborative groups, and class-wide discussions. The CSM program primarily 
used manipulatives during the CMRS course. Pattern blocks were used with fractions, 
multiplication and division of fractions, and detennining algorithms; base- 10 blocks were 
used to demonstrate division problems; number lines served as a tool for fraction 
division; and Cuisenaire Rods were used to demonstrate multiplication and division of 
fractions. Teacher D enjoyed the variety of manipulatives used because they expanded 
her thinking and Teacher I enjoyed manipulatives because she is “a visual learner so that 
help[ed] me to grasp concepts better.” Teacher E reflected: 

[I] loved the Cuisenaire Rods and the base ten blocks. These are ways the kids are 
learning math today and since I did not learn that way it can be hard for me to 
relate to it and help them when they are confused. I like learning new ways to do 
old things especially when it makes more sense the new way. 

CPM class labs used motion detectors and computers to simulate and graph motion, 
speed and velocity. “My favorite part was working with the motion detector. I liked that 
we were able to see the graph then try to create the same graph by experimenting,” wrote 
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Teacher O. PBL class incorporated Lego Mindstorms, and several classes used TI-83 
graphing calculators during work with functions, variables, and graphing. Teacher J 
summarized many participants’ sentiments when he wrote, “I enjoyed learning how to 
work the LEGO [.vie] robot and how to program it to do different things.” Collaborative 
groups of varying sizes were used throughout all CSM classes, as were class-wide and 
small-group discussions. Teacher P enjoyed collaborative groups because they made him 
feel successful and Teacher N felt collaborative groups “gave everyone a chance to voice 
their ideas” and she needed “that chance to verbalize to commit [concepts] to memory.” 
Teachers appreciated discussions because they “stimulated higher level learning” 
(Teacher A) and allowed “everyone [to feed] off ideas” (Teacher Q). 

The teachers also enjoyed learning via multiple solutions, conceptual teaching, 
and modeling or demonstrations. Multiple solutions involved listening to different 
groups explain their solutions, discussing various methods for solving problems, and 
learning how to teach concepts a variety of ways. Teacher R “liked seeing the different 
solution styles. It helped [her] know how the students might think about it.” Teacher A 
commented: 

I enjoy [ed] the discussion of the different ways that various problems were 
solved. I realized after solving a problem that another member had a “better” way 
of solving the problem .... [meaning] a way that I think students might better 
understand. 

Instructors modeled conceptual teaching by building concepts up from the concrete to the 
abstract. Participants were asked to draw pictures or representations of concepts, then 
model the concept with manipulatives, then discuss and verbalize them abstractly. 
Teacher Q “enjoyed the concrete to pictorial to abstract work. Explaining where all the 
parts of the picture relate to the numerical representations was very illuminating.” 
Reflecting on a CMRS lesson, Teacher C wrote: 

I loved revisiting division of a whole number by a Mixed [.v/c] number. I feel that 
I have a better understanding having drawn the pictorial models. Division is a 
difficult concept for a lot of students so I embrace any tools that may assist with 
teaching that. 




67 



Occasionally, instructors modeled a complete lesson for participants, other times, they 
modeled teaching small parts. Participants were especially appreciative after a CPM 
lesson was modeled; “we went through the lesson in a way similar to how it would be 
presented, including the specific handouts that would be used,” wrote Teacher R. 

Many participant teachers enjoyed any activity that involved problem solving. 

This was most often commented on in PBL class reflections in relation to the engineering 
design process as teachers worked in collaborative groups to design a Rube-Goldberg 
machine. “All the energy going into the design plan reinforces how motivating problem 
solving is. It was fun,” wrote Teacher Q. Comments on problem solving were not limited 
to PBL class reflections. Several teachers also positively reflected after all three courses 
on using manipulatives to understand problems, working practice problems individually, 
and discussing strategies for problem solving and teaching ideas in small groups. 

Teacher O “liked using the apples, rods, pattern blocks, etc. to work the problems .... 
[and] collaborating in our groups to discuss the different ways to work the problems.” 

Teaching methods that participant teachers appreciated the least or even 
adamantly disliked were less exploratory and more teacher-centered. In general, most 
teachers did not relish completing worksheets or taking notes. Teacher K, for example, 
felt that guided notes were a waste of time because he thought his students would only 
listen for the key words to fill in blanks. Teachers also did not enjoy direct teaching, 
especially PowerPoint presentations that were read verbatim. Teacher A “found 
[himself] drifting off and not paying as much attention as when the class was interjecting 
thoughts and examples” and Teacher E became “restless and [had] trouble focusing.” 
Additionally, many teachers commented that they are hands-on learners, do not leam 
aurally, and therefore grew bored and zoned-out during periods of direct teaching. 
Teacher N summarized the downside of direct teaching with her comment stating, “I 
heard people tell me how to use this information, but I didn’t get a chance to discuss or 
think about it on my own.” 

Perceptions of the classes. 

Many teachers were pleased with the CMRS class. Several teachers appreciated 
the CMRS instructor’s teaching style. Teacher C wished she “had more one-on-one time 
with [the instructor].” Some participant math teachers commented that they just enjoy 




68 



doing math period, whatever the context; “Yeah! I love math!” remarked Teacher G. 
Interestingly, no participant science teachers commented in kind regarding science. None 
of the attending science teachers however, taught physics alone, or had heavy physics 
backgrounds and much of secondary level physics is highly math-related. 

Many teachers commented that the math lessons were too short. “I only wish we 
had more time to work on Math [.m'c],” lamented Teacher C. Teacher O expressed 
frustration that some of the math content covered in the CMRS class was not aligned with 
the TEKS and wondered why it was included. Although use of mathematics 
manipulatives was highly appreciated, those teachers with little prior experience with 
manipulatives were more likely to become confused and could have benefitted from more 
independent practice time; this was especially true with Cuisenaire Rods. Teacher B 
wrote: 

I had trouble using the Cuisenaire Rods and considering I will be receiving a set 
to use in my classroom, I will need more time to develop a deeper understanding 
of how they can be used and in what ways and all the ways. 

Similarly, teachers with little prior experience with graphing calculators felt they did not 
have enough time to get comfortable with the calculators’ use and were frustrated when 
they could not keep up with activities that incorporated graphing calculators. Teacher M 
“felt that time was not given for people who are not familiar with the TI-83 and thus I did 
not like using the TI-83 before getting a little bit of formal practice with it first.” During 
review of math problems, several participants commented that they became lost because 
they did not have enough time to process a question and develop an answer before 
another participant called out the answer. 

Discussions held during CPM class sparked many complaints on daily reflections. 
Discussions, mostly related to but not limited to science, were too long, were not 
facilitated well, and content was sometimes above the comprehension level of teachers of 
lower grades. As a result, participants reported feeling excluded, angry, confused, 
frustrated, irritated, or zoned-out, and some commented that the discussions were a waste 
of time. A sample of comments on CPM discussions include: 

• “I felt that the discussion was way too long with no closure at the end” 
(Teacher H). 
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• “The discussion on reference point, etc. was not useful - it was not given in a 
style that I personally learn it. It might have helped if the instructor had found 
a way to focus the discussion, not allowing a few outspoken individuals to 
dominate” (Teacher R). 

• “The discussion went on too long - 1 got confused about what we were 
discussing!” (Teacher G). 

• “I still get a little frustrated during the physics lessons when we discuss a 
certain concept for a long time with no solid conclusion” (Teacher E). 

All teachers enjoyed the PBL class. They liked experimenting with designs and 
working in groups. However, the PBL class has room for improvement. Many 6 th - 8 th 
grade teachers complained that the PBL class, particularly the time allotted to work 
collaboratively on their project, was too short. “The class time was too short. Lor a 
project such as the one we are challenged to do we need more time,” wrote Teacher A. 
Some teachers felt this negatively affected the quality of their Rube Goldberg machines 
partially because they ran out of time to adequately test their designs. Teacher F 
commented that her team’s “invention [was] not working at the last minute. I felt we did 
not have enough time.” It might have been helpful to have more instructors available 
during group work time; several participants felt they did not receive enough one-on-one 
help from instructors. Participants also grew weary of multiple daily reflections and 
thought the reflections questions asked of the PBL class did not align well with the 
activities of the class. 

Emotional responses to CSM. 

Teacher reflections contained a wide variety of emotional responses. Through 
cross-case analysis, a number of recurring emotional response themes became clear. 
These include positive responses of feeling engaged, experiencing self-assurance and 
experiencing validation; responses of lacking background knowledge, feeling challenged, 
feeling confused, feeling frustrated, and wishing for elaboration, which relate to content; 
and negative responses of feeling anxiety, feeling empathy, feeling exclusion, feeling 
uncomfortable, and experiencing disappointment. 

The emotional response themes were analyzed via matrix-style pattern analysis 
(Bazeley, 2007) to determine whether response patterns varied by different demographic 
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groups. Demographic variables that were most helpful in understanding teacher 
perception differences were the level of content background knowledge (low or high 
science or mathematics), the grade level taught (5, 6, 7, 8, or a combination of grade 
levels), and the discipline taught (mathematics, science or both). 

Feeling engaged. 

Teacher participants felt more engaged in CSM classes when they were involved 
in hands-on activities such as using math manipulatives or Lego Mindstorms, and 
building projects or when they were able to contribute to class discussions. Teacher P 
wrote that using “manipulatives and measuring . . . keeps my attention to what is 
happening in class” and Teacher E commented that “doing the math activities keeps me 
more hands on and focused on the lesson.” Teacher O “liked learning how to program 
the robots because it was hands on and engaging” while the Lego Mindstorms really 
excited Teacher H who wrote, “We got to play with robots! I felt so empowered! They 
were easy and fun!” Teacher M “enjoyed the inquiry for explaining position/point of 
reference. It encouraged interaction and discussion as well as high level thinking of the 
students (us).” Additionally, Teacher A felt that being able to contribute to discussions 
“allows students to focus on what is going on in class better than just sitting and 
listening.” 

Interestingly, teachers with lower levels of content background knowledge 
commented on their level of engagement more frequently than teachers with high levels 
of background content knowledge. Also, 6 th grade teachers wrote the majority of 
engagement comments on their daily reflections; two-thirds of the individual comments 
relating to engagement were from 6 th grade teachers. 

Experiencing self-assurance. 

Experiences can bolster or depress one’s self-assurance regarding a topic 
depending on the degree of success experienced. Roughly equal numbers of comments 
relating to self-assurance were associated with bolstering experiences as depressing 
experiences. Participants’ self-assurance was bolstered when they were able to 
successfully complete activities (especially mathematics and PBL activities), and 
participate in discussions. Teacher K received a boost “when we were reviewing how to 
calculate distance and it reminded me that I could do the math” and Teacher D felt self- 
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assured when she “was able to enter into the discussion of speed due to prior knowledge 
taught about distance covered and time elapsed.” Participants’ self-assurance was 
depressed when they experienced content or an activity beyond their current level of 
knowledge, could not grasp content as quickly as they would have liked, and felt as if 
they should already know what was being tested or covered. In the words of Teacher G, 
“On questions that were at a 5th grade level, I felt confident. I really did feel STUPID 
[original emphasis] on questions above 5th grade.” In reference to the pretesting, 

Teacher L wrote, “I understand its [sic} component [s/c] to assessing background 
knowledge as a teacher. I disliked the uncomfortableness of not knowing knowledge I 
felt like I should have known as a teacher.” 

Demographic splits in self-assurance-coded comments are evident. Teachers with 
LSBK and LMBK commented much more frequently on their self-assurance than 
teachers with HSBK and HMBK. It is possible HSBK and HMBK participants did not 
see a need to comment on self-assurance as they were fairly confident in their content 
abilities. Science teachers commented on self-assurance at a much higher rate than 
mathematics teachers and teachers of both science and mathematics. As previously 
mentioned, an equal number of positive and negative self-assurance comments were 
associated with all levels of background knowledge and discipline taught. However, the 
number of positive and negative self-assurance comments were not evenly split across the 
demographic grade level taught. Fifth and 6 th grade teachers commented on self- 
assurance at higher rates than 7 th and 8 th grade teachers. The lower the grade taught by 
the teacher, the more likely the teacher commented on experiencing self-assurance 
depressors. Three-fourths of 5 th grade teachers’ comments on self-assurance were 
depressor events. Fifth grade teachers were most affected by content above their level of 
understanding before and during lessons. Sixth grade teacher comments equally 
addressed self-assurance boosters and depressors, but all depressor comments dealt with 
CPM content or discussions. Three-fourths of 7 th and 8 th grade teacher self-assurance 
comments addressed bolstering events. 

As a whole, participant teachers commented equally as often on experiences 
which both bolstered and depressed their self-assurance. Teachers with low levels of 
background knowledge and science teachers were more likely to reflect on self-assurance 
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than teachers with high levels of background knowledge and mathematics teachers, 
respectively. As the grade a participant teacher taught decreased, the degree of 
commenting on self-assurance increased, and the likelihood that an experience depressed 
self-assurance increased. 

Experiencing validation. 

Participants commented throughout the summer institute when various 
experiences validated their current mathematics and general teaching practices. Such 
comments were most frequently made by 5 th and 6 th grade teachers. Teacher I “liked the 
idea of showing your work with numbers, words and pictures. It makes sense to force all 
answers to be shown like that ... because I constantly re-enforce drawing pictures and 
showing work in my own classrooms.” Additionally when CSM instructors “were not 
satisfied with a partial answer but . . . kept probing until the student gave a complete 
answer,” Teacher A felt this “validated some of the things [he] like[s] to do in class.” A 
few validation comments from higher grade teachers also addressed learning that their 
work for the CSM class was on track or successful. For example, Teacher M was pleased 
to observe another “participant’s created lesson in action” because “it allowed [her] to see 
if the lesson [she was] designing [was] on track.” Mathematics teachers commented on 
receiving validation of their mathematics and general teaching practices two to three 
times more often than science and combination science and mathematics teachers. There 
were no differences in the rate of commenting on validation among teachers of different 
levels of mathematics and science background knowledge. 

Feeling anxiety. 

Some participants expressed anxiety in their summer institute reflections. The 
focus of the anxiety ranged from viewing worrisome or phobia-related video, to worry 
over upcoming tasks such as public speaking, group work, and beginning the design 
challenge, to feeling overwhelmed in general. Teachers with HMBK expressed anxiety 
comments at a lesser rate than teachers with LMBK, LSBK, and HSBK. The lower the 
grade taught by the participant, the greater the rate of comment on anxiety such that most 
comments were written by 5 th and 6 th grade teachers. Most comments might be li nk ed to 
personal apprehension such as “my nerves at talking in front of everyone and worrying if 
ours would work” (Teacher G) and “I chose to use the motion detector for my portion of 
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the correlated lessons and am rather self conscious [sic] of the task” (Teacher B). 
However comments by two 5 th grade teachers might indicate that the range of grade 
levels represented in the professional development is too large. For example. Teacher L 
was “afraid of being in a group again. I have always felt unaccomplished in groups” and 
Teacher G wrote: 

I am feeling very overwhelmed with a project. I barely know (or understand) 

different forms of energy. I am feeling a little directionless and don’t quite know 

what to do next. I cleared that up at the end, but I am feeling like the little 5 th 

grader running around with big kids! 

Lacking background knowledge. 

All participants learned content and/or teaching strategies and this will be 
addressed in a later section; however, some participants additionally thought to reflect 
when they specifically had not had experience with certain content or activities. Teachers 
felt they lacked background knowledge in mathematics including general mathematics, 
fractions, number properties, and the Cartesian plane, and in science including general 
physics and science vocabulary. They also commented that they were lacking experience 
with or knowledge of mathematics manipulatives, Rube Goldberg devices, and CSM 
technologies including robotics, graphing calculators, and motion sensors. A small 
sample of reflections regarding lack of background knowledge includes: 

• “I've never been exposed to number lines with regard to fractions and it took 
some thinking to make the two relate.” (Teacher N) 

• “I did not remember the properties/rules and all the algebraic reasoning.” 
(Teacher L) 

• “We went through a lot of math concepts and vocabulary that I haven’t used in 
years and we went through it very quickly. I was lost anytime people started 
talking about irrational numbers and rationals and associative and so on.” 
(Teacher E) 

• “I had trouble on the physics test but it is understandable because I don’t have 
the background knowledge to grasp the concepts very well.” (Teacher J) 

• “I did not know about the different between independent and dependant 
variables. I thought I did.” (Teacher H) 
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• “I haven’t had a lot of experience using the manipulatives so it was interesting 
to see a lesson utilizing them.” (Teacher N) 

• “I have never used robotics, however have read a lot about them. I am pleased 
to be able to begin working with them.” (Teacher R) 

• “I had never heard of Rube Goldberg before and it was an interesting idea.” 
(Teacher C) 

More comments on lack of background knowledge were written by teachers with 
LMBK and LSBK, but when corrected for the number of participants in each 
demographic, LMBK, LSBK and HSBK teachers commented at similar rates and HMBK 
teachers commented at the highest rate. The comments of HMBK teachers focused 
mostly on lack of knowledge or experience with Rube Goldberg devices and technology 
and to a lesser degree on science vocabulary and content. The focus of HSBK teacher 
comments was largely the same as that of HMBK teacher comments except that there 
were more comments reflecting lack of mathematics knowledge. Comments by LSBK 
and LMBK teachers focused largely on lack of content and vocabulary knowledge, and 
lack of experience with manipulatives and robotics. 

When comments regarding lack of background knowledge are split by the 
discipline taught, patterns emerge as one might expect. Mathematics teachers and 
teachers of both mathematics and science commented most often about lacking science 
content and science teachers commented most often about lacking mathematics content. 
Lack of experience with technology was the second-most commented area for all three 
sets of teacher disciplines, followed by lack of experience with Rube Goldberg devices. 
Science teachers wrote about lack of background knowledge at the highest rate. 

Lower grade teachers reflected on their lack of background knowledge at higher 
rates than higher grade teachers. Teachers of all grade levels expressed little or no 
experience with the technologies used in the CSM summer institute. Lack of technology 
experience was predominantly what 7 th and 8 th grade teachers reflected on. 5 th and 6 th 
grade teachers also lacked content knowledge in science, mathematics, and vocabulary, 
and admitted not knowing about Rube Goldberg devices. 

In summary, teachers with low content background knowledge and teachers of 
lower grades were more likely to reflect on lack of background knowledge in science and 
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mathematics content and vocabulary. Those with high levels of background knowledge 
or taught higher grades were more likely to reflect on lack of experience with technology 
and Rube Goldberg devices. Teachers of science were most likely to reflect on their lack 
of mathematics content and teachers of mathematics were most likely to comment on 
their lack of science content. 

Feeling challenged. 

Teachers felt challenged by aspects of the CSM summer institute. Challenged can 
have two polar connotations. A positive connotation of challenge means that participants 
were stretched to expand their knowledge and enjoyed doing so. A negative connotation 
of challenge means that participants struggled with an aspect of the program. Most of the 
challenge comments related to mathematics content, mathematics manipulatives, and 
science content. Participants who struggled had difficulty with fractions, the CPM 
pretest, acceleration, and the number line and pattern block manipulatives. For instance, 
Teacher K had difficulty trying to “figure out fractions” and felt that irregular fractions 
were “just harder even with the [pattern] blocks.” Participants who enjoyed a stretch 
exercise were challenged by the mathematics pretest, fractions, and number line and 
pattern block manipulatives. Teacher I “actually enjoyed the [CMRS] test ... [.vie] some 
questions were information I once learned but have forgotten ... [sic] I guess that's my 
favorite part ... [sic] was being challenged again.” Teacher F found that “using the 
manipulatives and number line to divide .... was challenging, new and interesting.” 

Participants with LSBK or LMBK wrote that they were challenged at a much 
higher rate than HSBK and HMBK participants. The splits between positive and 
negative challenges however were close to even for all categories of the background 
knowledge demographic. All three disciplines’ categories of teachers also had even splits 
of positive and negative challenges. Both positive and negative challenges reflected upon 
by science teachers and teachers of both mathematics and science span all discussed 
categories - mathematics content, mathematics manipulatives, and science content. 
Mathematics teachers’ positive challenges were dominated by mathematics content and 
their negative challenges spanned the three categories. 

Different patterns emerged for grade levels. Lower grade teachers commented 
more frequently on challenges than teachers of higher grades. The balance of positive to 
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negative challenges also changed with teacher grade level. 5 th grade teachers’ challenges 
were all struggles relating to mathematics content, manipulatives, and science content. 

6 th grade teachers wrote of more positive challenges than negative challenges; some 
enjoyed manipulatives and mathematics problems both in classes and on the pretests, and 
others struggled with mathematics content and manipulatives. Two comments by 7 th 
grade teachers reflected negative challenges with mathematics manipulatives and physics 
content. Two comments by 8 th grade teachers reflected positive challenges with 
mathematics manipulatives. 

In general, participant teachers experienced struggles and enjoyable challenges 
with the program content equally as much. Math teachers wrote of more enjoyment of 
mathematics challenges. Teachers of lower grades struggled more than teachers of 
higher grades with content and manipulatives. 

Feeling confused. 

Confusion with content or activities was expressed by many participants in their 
daily reflections. Content areas that most confused teachers included: fractions, 
functions, discussing multiple solutions in mathematics class, mathematics manipulatives 
(Cuisenaire Rods and pattern blocks), and major CPM discussions especially on frame of 
reference and displacement. Teacher G “got a little lost in the fractions conversations” 
and Teacher E did not enjoy “doing functions and all of that because the lesson was way 
too quick for me as a non math person.” Teacher E later commented that “for me it is 
actually harder sometimes to see all the different ways to do one problem because when I 
actually understand one way all the others confuse me.” Additionally, Teacher N “did 
not like the [Cuisenaire] rods. I felt like they made the problem more difficult and 
confusing than it needed to be.” Comments indicating confusion also indicated that 
teachers felt that content was over their heads. Teacher G became confused “any time we 
went past (like way past) [her] knowledge level” and at least one problem set was not 
productive according to Teacher A because “we lost about half of the class doing a 
problem at a level not appropriate to the teachers comfort level.” Teachers were 
confused when they did not have enough time to digest and understand the content. 
Teacher Q desired more time to learn manipulatives and had difficulty “understanding the 
last thing that [was done] with the rods, laying them on top of each other and switching 
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one rod out for smaller pieces. It was the end of the class, and we had to move on.” 
Finally, teacher confusion was exacerbated when lesson topics were switched too rapidly. 
“I had trouble understanding using and going between several sets of manipulatives. I got 
lost,” wrote Teacher H. Similarly, Teacher D got “lost in the transition points during the 
science parts of the lesson” and felt that “constant changing from one manipulative to 
another made it difficult to follow the lesson.” 

Comparing confusion reflections based on content background knowledge 
revealed that teachers with LMBK commented on confusion at the highest rate, teachers 
with HMBK commented at the lowest rate and teachers with LSBK and HSBK 
commented on confusion at about the same, intermediate rate. HMBK teachers were 
least confused about mathematics content and manipulatives and all discussions. HSBK 
teachers were least confused about science content. 

Fifth grade teachers wrote about being confused at a much higher rate than 
teachers of all other grades. They had problems with all areas listed above, but especially 
mathematics content, manipulatives, and science discussions. Sixth through 8 th grade 
teachers commented about confusion at similar rates, but 7 th and 8 th grade teachers were 
not confused with science content and had fewer problems with mathematics and 
manipulatives. 

Science and mathematics teachers did not differ in their rate of commenting on 
confusion. There was a difference in the things each groups tended to be confused about. 
Although both science and mathematics teachers were confused in the CMRS class, 
science teachers had more trouble with the mathematics content and mathematics 
teachers had more trouble with the manipulatives. Science teachers did not write of 
being confused in CPM class, but mathematics teachers did and many had difficulty 
following the lengthy science discussions. Science teachers were more likely confused 
when multiple solutions to problems were discussed or presented. 

In general, participants with lower levels of content knowledge and teachers of 
lower grades experienced confusion more often than participants with higher levels of 
content knowledge and teachers of higher grades. Science teachers were more apt to be 
confused by mathematics content and mathematics teachers were more apt to be confused 
by science discussions and mathematics manipulatives. 
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Feeling frustrated. 

Teacher participants wrote of feeling frustrated during the CSM summer institute. 
Teacher G even asked, presumably in jest, “Is there a frustration tek [s/c]?” The majority 
of comments indicating frustration centered on three areas: mathematics content, science 
discussions, and working with technology. Those frustrated in CMRS class struggled 
with fraction and number line problems, and understanding the concepts conceptually. 
Teacher P “did not enjoy the frustration level when [he] did not comprehend the initial 
fraction exercises” and Teacher H “really struggled with the number line piece and would 
have liked more time to explore.” The CPM discussions, especially one focused on 
frames of reference, riled up and frustrated teacher participants because it went on too 
long, without clear direction or sense of closure. In the words of Teacher E: 

I did not enjoy the questions after questions that were never ending. I think the 
discussion went on WAY TO LONG [original emphasis]. I understand the point 
was to make us think. . . however for me it was very frustrating. We discussed 
what a reference point was for over an hour and then finally the answer we all 
HAD BEEN SAYING [original emphasis] . . . was determined. . . and then that was 
it. 

Activities utilizing technology thwarted different teachers at different points. Individuals 
reported having trouble using Apple brand computers and the associated graphing 
program, motion detectors, and Lego Mindstorms; it took some teachers more time than 
they desired to figure out these technologies. According to Teacher G, “Apple computers 
are a little frustrating for me. They don't click where I want them to click.” Teacher E’s 
PBL team became frustrated with the Lego Mindstorms when they “could not figure out 
how to get the brain to talk to us even though [they] worked on it for 30 minutes.” 

There was a distinction in the rate of comment on frustration between participants 
with low content background knowledge and high content background knowledge. 

LSBK teachers commented twice as frequently as HSBK teachers and LMBK teachers 
commented one and one-half times as frequently as HMBK teachers. Teachers with 
LSBK or LMBK were frustrated by all of the main areas discussed above. Participants 
with higher content background knowledge reported much less frustration with 
mathematics content and somewhat less frustration with science discussions. Their 
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frustrations leaned more towards how the summer institute was facilitated rather than 
whether or not they could do the work. 

There was a disparity in both the rate at which teachers of different disciplines 
wrote comments reflecting frustration and the aspects of the summer institute they found 
frustrating. Science teachers commented on frustration at a rate 3.5 times greater than 
math teachers. Science teachers had the most trouble with mathematics content, 
complained most about the science discussions, and commented most about technology 
issues. Mathematics teachers were frustrated by those areas to a much lesser degree and 
some were additionally frustrated by the CPM pretest and group work. 

Review of frustration comments by grade level taught reveal that 80% of the 
comments were from 5 th and 6 th grade teachers. Teachers of both 5 th and 6 th grades 
commented at similar rates and the subject of the comments spanned the three main 
areas: mathematics content, science discussions, and technology usage. 5th grade 
teachers wrote most frequently about trouble with mathematics content and were the only 
teachers to be frustrated with science content. Sixth grade teachers were strongly 
frustrated by mathematics content, science discussions, and technology problems. The 
smattering of frustration comments from 7 th and 8 th grade teachers focused on 
mathematics content and program facilitation. 

In summary, teachers with LSBK or LMBK, were more likely to express 
frustration with content aspects of the CSM summer institute. Lower grade teachers were 
much more likely to express frustration with mathematics content, science discussions, 
and technology use than higher grade teachers. Science teachers were more likely to 
express frustration than mathematics teachers especially regarding mathematics content 
and science discussions. Teachers with higher levels of background knowledge and 
teachers of higher grades expressed the least frustration and such comments regarded 
facilitation of CSM more than content aspects. 

Wishing for elaboration or enhanced material. 

In addition to reflecting on aspects of the CSM summer institute that they had 
difficulty with, participant teachers clarified what they would have liked to have received 
more information on during the institute. As with other aspects of teacher perception, 
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rates of response and topics of response varied with demographic group. A wide variety 
of topics were referenced by the participants’ desires. 

Teachers with LMBK requested elaboration on content more often than teachers 
with HMBK. LMBK teachers indicated that they did not understand a concept or did not 
have enough time to work with and absorb the concept. Examples include fraction slants 
(“writing fractions with slanting lines vs [sic] straight lines”), how the robots work, and 
how to explain concepts via manipulatives. Additionally, LMBK teachers wished to 
know which TEKS go with which activities and indicated they needed to learn the new 
science TEKS. Teachers with HMBK commented on aspects that might best be 
classified as extensions to the CSM curriculum. Such teachers desired further 
information on appropriate children’s literature to incorporate into their classes, the 
development of measurement systems, PowerPoint tips, and a side-by-side comparison 
chart for science and mathematics TEKS. 

Mathematics teachers were considerably more vocal about what they wished had 
been included in the CSM summer institute than science teachers such that comments 
from mathematics teachers were 76% of all comments recorded. Mathematics teachers 
were interested in additional infonnation on fractions, TEKS, and CSM extensions. 
Science teachers wanted more directions and information on how the mathematics and 
science TEKS correlated, and some teachers wrote reminders of what they needed to 
learn on their own time after CSM. 

Differences by grade level taught were similar to differences by level of content 
background knowledge. Fifth grade teachers wrote of their desire for more information 
at the highest rate, and both 5 th and 6 th grade teachers wanted more information on 
content and how TEKS aligned with each lesson and what gaps or misconceptions 
existed for the TEKS. Eighth grade teacher comments mostly fell into the extension 
category as they wished for a TEKs comparison chart, recommendations for children’s 
literature and additional example mathematics lessons. Teachers from all grades 
additionally listed reminders of topics they wanted to leam more about on their own at a 
later time. 

Teachers of higher grades and teachers with HSBK or HMBK were more likely to 
request more information on CSM extension material while teachers of lower grades and 
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teachers with LSBK or LMBK were more likely to want more infonnation on content and 
TEKS. Mathematics teachers were much more vocal about their desires than science 
teachers. 

Feeling empathy. 

Participant teachers thought about their students throughout CSM, and some 
teachers’ daily reflections expressed empathy for their students. This empathy was 
expressed in two general forms. Teachers expressed feeling like a student themselves 
when they were lost in a CSM class or frustrated with a teaching strategy. For example, 
Teacher Q was lost during one CPM class and wrote, “[the instructor] was explaining at 
the end and [I was] not quite getting it. It was the fact that the lesson was ended before I 
understood. [This] makes me a bit more empathetic to my students.” Teacher E similarly 
commented after a CMRS lesson that: 

never having used pattern blocks before and not doing fractions in a long time I 
had no idea what to do. I imagine this is how the kids would feel and being given 
a task with absolutely no idea how to do it was kind of frustrating. I understood 
after someone else was able to explain it to me. 

Other teachers were reacquainted with student frustrations when working with groups 
and limited equipment. Teacher F reflected that “it is now easier for me to understand 
when my students complain when they are working in groups, it is difficult to 
collaborate,” and Teacher P commented that he empathized with “the frustration students 
can experience when having to share equipment.” 

Teachers also expressed consideration for how students think about concepts or 
would react to a particular activity or teaching strategy. Learning to incorporate pictorial 
models into mathematics lessons helped Teacher J “to explain and understand the 
problem better and ... to realize what the students are thinking.” Teacher A thought that 
“the motion detector was difficult to use and produce the requested graphs. This could get 
frustrating and cause students to quit paying attention or to not finish.” 

Teachers did show differences in the rate of comment on empathy by 
demographic group. Teachers with LSBK wrote about empathy at twice the rate of 
teachers with HSBK; there was no difference in the proportion of empathy forms written 
about. Only one teacher with HMBK wrote an empathetic comment, and instead of 
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relating to students, it depicted empathy to other teachers in the program. Math teachers 
expressed empathy in their reflections at half the rate of science teachers and teachers of 
both science and mathematics. As the grade a teacher taught increased from 5 th to 8 th 
grade, the rate at which the teacher reflected on empathy decreased. Also, lower grade 
teachers were more likely to express feeling like a student themselves, while teachers of 
higher grades reflected in a more balanced manner on all forms of empathy. In general, 
the type and degree of empathy expressed by a participant changed from feeling like a 
student and high rates of commentary to a balance between feeling like a student and how 
students might react or think about an activity and lower rates of commentary as the 
grade level a teacher taught increased and a teacher’s amount of background content 
knowledge increased. 

Feeling excluded. 

A few participants expressed feeling excluded from CSM class discussions. All 
such comments were written by 5 th and 6 th grade teachers. The 5 th grade teachers all 
commented that these discussions were over their head. For example, Teacher Q “was 
not able to grasp what some folks [were] talking about in the discussions.” The 6 th grade 
teachers felt left out because the discussions were active only amongst a few people and 
most of the class was drifting off. Teacher E “was not able to stay focused on a 
conversation between 3 or 4 people.” “A lot of people felt left out and disconnected,” 
wrote Teacher F. The result of both complaint types was that these teachers were not 
able to contribute to the discussions. Additionally, one 6 th grade teacher, Teacher C, felt 
excluded during other class activities, but this was due to her missing the first week of the 
summer institute. 

Feeling uncomfortable. 

A small number of participants expressed feeling uncomfortable during the CSM 
summer institute. Only 5 th and 6 th grade teachers expressed uncomfortableness in their 
reflections. Teacher L did not like that she did not have a good knowledge base for the 
content covered in CSM. She had not experienced being behind in class before and had 
trouble handling her uneasiness. Two other teachers were not comfortable during a CPM 
class discussion on point of reference. There was a bad vibe in the room and the 
discussion was difficult to follow. Teacher N wrote: 




83 



There was a lot of shouting out and I couldn’t understand what anyone was 
saying. It felt like people were defensive about their ideas and it made me 
uncomfortable to say anything because I wasn’t sure what the response would be. 
Experiencing disappointment. 

Teachers expressed disappointment with certain aspects of the CSM summer 
institute. The majority of disappointments related to the PBL component of the institute. 
Teachers were disappointed they did not have more time to work on their projects, and 
several were disappointed in the outcomes of their projects - the projects did not work as 
they envisioned. “I did not like that our machine did not work on the first try,” wrote 
Teacher A. Teacher D expressed disappointment at “not being able to correct the 
problem on our device. If we had more time and or more collaboration I believe it would 
have worked.” Teacher O was upset that she did not have the resources to implement the 
PBL project in her class. Additionally, Teacher M was disappointed she did not get to 
work on the PBL activity with her school partners, and Teacher H was disappointed there 
was little time allocated during CSM to discuss in detail vocabulary disconnections 
between mathematics and science. Interestingly, most of the comments regarding 
disappointment (67%) came from mathematics teachers. 

Effect of CSM on participant teachers’ content knowledge and pedagogy. 
Two core features of effective professional development are a focus on content 
and inclusion of active learning (Binnan et ah, 2000; Desimone et ah, 2002; Garet et ah, 
2001). Investigating whether and to what degree participant teachers learn content and 
pedagogy help detennine how effective a professional development CSM is (Guskey, 
2000). Data regarding teacher learning was gathered from content tests and teacher 
reflections, observations and interviews. 

Results of teacher pretests and posttests. 

Participant teachers were administered pretests and posttests in physics and 
mathematical reasoning at the beginning and end of the two- week summer institute. A 
related-samples t-test was conducted for each content test to detennine whether teacher 
content knowledge of physics and mathematics significantly improved as a result of the 
summer institute. Results indicate that participant teachers did significantly improve 
their knowledge of both mathematical reasoning and physics. The mean score on the 
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mathematical reasoning posttest (M= 82.00, SD = 12.817) was significantly higher than 
the mean score on the mathematical reasoning pretest ( M = 73.50, SD = 13.135, /( 1 5) = 
2.984, p = .005). The effect size was medium (d = 0.746), and 37% of the variance was 
accounted for by the treatment (r 2 = 0.372). The 95% confidence interval for the mean 
difference between pretest and posttest mathematical reasoning scores was 2.43 to 14.57. 
The mean score on the physics posttest ( M = 62.29, SD = 12.970) was also significantly 
higher than the mean score on the physics pretest ( M = 45.76, SD = 1 1.377, t( 1 6) = 4.992, 
p = .000). The effect size was large ( d = 1.21 1), and 61% of the variance was accounted 
for by the treatment (r 2 = 0.609). The 95% confidence interval for the mean difference 
between pretest and posttest physics scores was 9.5 1 to 23.55. Although it would be 
interesting to also understand whether significant differences between pretest and posttest 
scores on both content tests exist between sub-sample demographic groups such as 
teachers of one grade or discipline, or teachers with certain amounts of college science or 
mathematics training, the sample sizes of these sub-samples were too small to generate 
valid t-test results. 

Participant teacher gains in specific content and content pedagogy. 

Mathematics. 

Participant teachers reflected on a great deal of mathematics concepts and 
pedagogy strategies they gained a better understanding of during the summer institute of 
CSM. Teachers learned about fractions, the Cartesian plane, mathematical properties, 
and proportional reasoning as well as how to incorporate mathematical manipulatives 
into the teaching of these concepts. 

Much time at the summer institute was spent on fractions. This benefitted a 
number of participants, many of whom reported needing a refresher on basic fraction 
concepts. “I had forgotten how to multiply and divide fractions,” wrote Teacher G. 
Teachers commented that they learned or were reminded how fractions and ratios differ 
(Teacher L “never thought of a ratio as a fraction”), how to work with mixed fractions, 
and how to multiply and divide fractions, including “why we multiply [by] the reciprocal 
to divide fractions” (Teacher M). 

Teachers picked up many teaching strategies for fraction concepts as well. 
Teachers learned how to demonstrate the butterfly method of comparing fractions and 
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how to show fractions as part of an array. Several types of mathematics manipulatives 
were introduced as concrete ways to teach fractions and other topics. Several teachers 
had no experience with mathematics manipulatives and many teachers commented on 
manipulative teaching ideas they had not seen before. “I learned about teaching fractions 
more conceptually and making sure students understand how it works,” reflected Teacher 
J. Teachers received practice using pattern blocks to model and explain the basics of 
fractions (fractions are parts of wholes), common denominators, fraction multiplication, 
and fraction division. Teacher G wrote that she “had never thought of changing the value 
of each pattern block to force students [to] see it as parts of a whole” and that she might 
use this exercise “towards the end of the lesson as an informal assessment.” Teachers 
learned how to use Cuisenaire Rods to model fraction sizes, relationships, and division. 
According to Teacher N, “Cuisenaire rods and fractions really do show the process of the 
size and relationships.” They also learned how to model fraction multiplication and 
division with a number line and how to model common denominators using triangle 
paper. 

Mathematical properties (associative, commutative, distributive, and closure) 
were reviewed during the CSM summer institute. At least half of the teachers reflected 
that they appreciated the review of this topic as they had forgotten or were unaware of 
some of the properties covered. The closure property was the least familiar to teachers. 
Teacher P learned “the differences between the different types of properties.” “I’m 
beginning to connect the terminology in math (associative, distributive) to the actual 
work with manipulatives which represent solutions to problems,” wrote Teacher Q. 
Teacher I “learned a different way to look at how to teach the properties MORE [original 
emphasis] effectively in my classroom.” Additionally, Teacher N reflected that the 
memory devices provided for mathematical properties “would help my students 
remember them.” 

Unit conversions and scale factor were proportional reasoning concepts teachers 
reflected on better understanding after the CSM summer institute. Teachers discussed the 
theory of converting units and learned how to perform conversions both by hand and 
using a graphing calculator. This benefited at least two teachers who commented that 
they now understood how conversions work. For example, Teacher Q wrote, “The 
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conceptual conversion lesson helped me understand how it works, rather than doing it 
with cross multiplication because that’s how I learned it. The broader discussion about 
teaching conceptually rather than procedurally was very helpful.” Several teachers 
reflected on pedagogy tips they learned regarding scale factors. One tip was “to use the 
outline of the one when showing the scale factor in a proportion” (Teacher F) and the 
other tip, from Teacher D, was to use hexagon pattern blocks to demonstrate scale 
increasing or decreasing. 

Many teachers benefited from the lessons on the Cartesian plane and graphing. 
Several teachers commented that they had not known the development history of the 
Cartesian plane and thought using a picture book to introduce the topic was a good idea. 
“Knowing the history can add depth or breadth to a student’s knowledge,” commented 
Teacher M. Teachers learned or were refreshed on several concepts including the 
definitions of: abscissa and ordinate, dependent and independent variables, scalar and 
vector numbers, functions and non-functions, and the distance formula. Teacher I 
“learned the correct vocab [s/c] for x and y,” Teacher K “was able to clarify what scalars 
and vectors were” and Teacher O “learned more about functions and non- functions and 
independent and dependent variables.” Additionally, teachers learned how to graph 
functions using a graphing calculator and computer software. Teachers also garnered 
strategies for teaching polar coordinates. 

Teachers learned new mathematics content and teaching strategies from the CSM 
summer institute. Content and pedagogy reviewed covered fractions, mathematical 
properties, proportional reasoning, the Cartesian plane, and other topics. Teacher E 
learned “new tools to do certain math concepts that I can do in my head or on paper but 
could not actually show before.” According to Teacher Q, “drawing the pathway out, 
applying fonnulas to a concrete example, [and] using different fonnulas to arrive [at] an 
answer gave me a better grasp of the math.” Teaching a concept pictorially and 
concretely before moving into the abstract was stressed in the CSM mathematics class 
and learned by many teachers. Teacher I “learned that doing the math without the 
algorithm makes your students truly think about what they are doing! Add [algorithms] 
and it makes the concept even more understandable and teachable.” 
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Science. 

The CPM lessons of the CSM summer institute covered a wide range of topics of 
which many concepts were unclear or unfamiliar to participant teachers. Recall from the 
analysis of participant transcripts that only 4 of 1 8 participants completed any college 
physics courses. Teachers commented on learning basic concepts on the scientific 
method, the distribution of natural resources, systems, motion, simple machines and 
energy transformations. 

The scientific method and types of investigations was discussed in detail at the 
summer institute because the science TEKS were being revised to incorporate 
comparative, descriptive, and experimental investigations. Many teachers were aware of 
neither the upcoming changes to the science TEKS nor the three types of investigations 
being added. Many teachers wrote about these investigation types in their reflections and 
commented that prior to attending CSM they were not aware that there is no single 
scientific method as is often taught in school. Teacher E “liked going over the 
experimental vs. comparative investigations and looking at the differences and learning 
the correct language” because she “obviously had some misconceptions and [was] glad to 
be on the right track correcting them.” However, not all teachers completely understood 
the differences and changes by the end of the CSM summer session; Teacher A wrote 
that he “thought it was interesting the Scientific Method [.vie] was no longer The [.vie] 
way to do experiments.” Additionally, a few teachers did not retain the knowledge on 
types of scientific investigation through the school year; after observing Teacher B in the 
spring, researchers noted that she did not “understand the scientific research designs and 
was surprised to learn” of them. 

One lesson during CPM class dealt with the distribution of land and water on the 
earth’s surface. This lesson was eye-opening for many teachers as they commented that 
they had never known the ratio of land to water on earth or the percentage of freshwater 
available for human use. Reflecting on “the amount of limited usable resources, [Teacher 
F] only had a vague idea of what all of those ideas really meant.” Many teachers thought 
they could adapt the activities from this lesson to their own classrooms. They liked the 
idea of using base 10 blocks to scale down and compare large quantities such as land and 
water mass. They also latched onto using a graduated cylinder to show percentages of 
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different things such as potable and non-potable water. Teacher R “liked the use of 
fractions in the world's water discussion .... [It was a] good opportunity to show division 
of fractions.” 

The concept of a system is important in the science TEKS and hence systems 
were covered in the CSM science class. In general, teachers came to class with a poor 
understanding of what a system is and how broad a concept it can be. Many teachers 
commented on learning the definition of a system, that everything is a system, and that 
one system can be composed of multiple other systems. Teacher I reflected on her 
enlightenment stating that “it made so much more sense when we learned that an ATOM 
[original emphasis] was a system ... if an Atom [s/c] is a system than everything must be 
a system because everything is made up of atoms.” Teachers also reflected on learning 
the difference between living and non-living systems as well as open and closed systems. 
Teachers may transfer the group activity done in the systems lesson to their class; 
Teacher G “liked the idea of presenting various dissimilar objects for students to explore 
definitions of systems.” Additionally, Teacher E wrote, “teaching systems is a huge part 
of what I teach so it is always nice to have new activities that can help students better 
understand the information.” 

Most teachers commented about learning the basic concepts of motion. They 
reflected on learning the differences between accuracy and precision; frame of reference 
and point of reference; distance and displacement and how to calculate both; and speed, 
velocity and acceleration and how to calculate and graph all three. Teacher P “learned 
how to describe position and frame of reference,” Teacher Q learned “the difference 
between distance and displacement .... and was unfamiliar with those terms before” and 
Teacher O “learned the differences between speed, velocity, and acceleration.” Several 
teaching strategies were demonstrated and favorably remembered by teachers. An 
activity involving a bug on a white board was used to introduce the concepts of distance 
and displacement and was enjoyed by several teachers. Teachers learned how to use 
motion detectors with computerized graphs to teach interpretation of motion graphs. 
Teacher P learned “how to design better labs with speed, distance, and time.” 
Additionally, teachers realized that they need to plan ahead of time what they want to 
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discuss about a topic. “I should really think about what I’m going to say before I begin 
explaining to make sure that it’s correct information,” reflected Teacher K. 

The PBL class allowed teachers to discuss and experiment with simple machines 
and energy transfonnations as they developed Rube Goldberg devices. Most teachers 
commented that they had no prior experience with Rube Goldberg machines and 
everything they learned about them was new. “I learned the basics of how a Rube 
Goldberg device worked. I have not previously had experience with them,” wrote 
Teacher J. Teachers learned about the three classes of levers, but some teachers were 
nonplussed to learn that “even though we teach 1 st , 2 nd , [and] 3 rd class levers in 7 th grade, 
they are never referred to again in the education of our students” (Teacher M). Teachers 
learned or reviewed the types of energy and how energy can be transformed from one 
type to another. The PBL Rube Goldberg activity was exciting for and left a mark on 
many teachers. According to Teacher N, “I could see how everyday problems could be 
used to illustrate science. It seemed like a great way to teach kids about energy transfers 
and simple machines because it was really concrete and relevant.” 

Teachers garnered much new science content and a few teaching strategies from 
the CPM lessons of CSM. Content covered an expanded view of the scientific method, 
the distribution of natural resource, systems, motion, simple machines, and energy 
transformations. Teachers came to CSM with low levels of physics background 
knowledge and therefore had much room to increase their knowledgebase. Fewer science 
teaching strategies were commented on than were mathematics teaching strategies 
although a few ideas struck teachers as useful. 

Technology. 

Technology was incorporated into portions of all three CSM classes. Calculators 
were used in CMRS, motion sensors and computers were used in CPM and Lego 
Mindstorms and computers were used in PBL. Teachers commented that they had little 
experience with each of these technologies, learned how to use each and gathered ideas 
for incorporating them into their classrooms. Teachers learned how to convert units and 
graph functions on the graphing calculators, how to use motion sensors and what kind of 
lessons they might prepare with them, and how to use and program Lego Mindstorms. 
Teacher O “learned how to convert measurements using the calculator” and Teacher J 
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“learned how to use the graphing calculator to show functions.” Teacher E learned “how 
to use the Lego Mindstonns software and robots.” In addition, some teachers commented 
that they learned how to incorporate PowerPoint in lessons and what the engineering 
design process entailed. Several teachers thought they might use robotics with their own 
classes. “The robots could be used in math [sic} .... with degrees and angles,” 
commented Teacher N. 

Language. 

Throughout the CSM institute and Saturday sessions, CSM instructors discussed 
how similar language that is used differently between disciplines can cause confusion for 
students. As part of the correlated model, it is important that teachers use the proper 
language of a discipline, identify parallel ideas between disciplines, and identify language 
that is confusing to students (West & Browning, 2010). Teachers did become more 
aware of the confusion language creates and the need to be more precise in its use. 
Teacher M “learned that Math [szc] has a lot more vocabulary similar to science with 
different meanings. [She] had always assumed that what they meant in science they 
meant in math if math used them.” Teacher B wrote, “I at times can be sloppy with my 
language and am guilty of assuming that students comprehend subjects only because of 
their ability to use vocabulary.” Teacher J “learned how using specific vocabulary and 
not assuming what the students know or don’t know is very important to dispel wrongful 
thinking.” Teachers noted that they hoped to work with their opposite discipline teacher 
partners to identify vocabulary overlaps and prevent student confusion. Teacher R liked 
the idea of creating a parallel dictionary as students go through classes. Other teachers 
thought they would think about language during planning and Teacher M decided to start 
adding important vocabulary words to her lesson plan to ensure she covered them 
correctly. 

Inquiry. 

One goal of the CSM professional development program was to increase the use 
of best practices in teaching science and mathematics and one such best practice, which is 
also a facet of correlated lessons, is inquiry. Most participant teachers’ understanding of 
inquiry expanded during the CSM program. They learned how inquiry differs from the 
scientific method and how to develop better inquiry activities. Teacher A was heartened 
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to receive reinforcement “that Discovery [sic] is a very good way to encourage learning.” 
Teacher B reflected that “inquiry method is not as restrictive as [she] thought but [she 
was] adjusting [her] own thinking” and felt she needed to review her current set of 
lessons “to see if I am doing inquiry or just kidding myself.” However, although some 
teachers did leam to differentiate inquiry from the scientific method, reflective comments 
of several other teachers indicate that they confused or equated inquiry with the three 
types of experimental design discussed during the lesson segment on scientific method. 
For instance. Teacher Q wrote “inquiry is a spectrum of approaches. We discussed 
design, experimental, and comparative.” And Teacher D reflected what she had to “re- 
orient her thinking about inquiry and the different types of inquiry depending on subject 
matter . . . [and] the range of inquiry that can be done from experimental to descriptive.” 

Understanding of integration. 

Participant teachers came to CSM with a variety of conceptions regarding 
integration and integrated teaching strategies. At the beginning of the CSM summer 
institute, teachers completed a survey entitled Satisfaction with Integrated Lessons (see 
Appendix A) and their pre-CSM conceptions were identified from this instrument. The 
teachers expressed many of the theoretical justifications purported in the literature. 
“Science and math go together” (Teacher K) and “the two disciplines seem to 
complement each other well” (Teacher N). Teachers agreed that integration increases the 
relevancy of content to students and that students transitioning from self-contained 
elementary classrooms to discipline-specific middle school classes, in the words of 
Teacher R, “need someone to help connect the concepts for them as a transition to high 
school.” Teachers have used and understand integration in different fashions - 
integration as interdisciplinary, mathematics as a tool, and science as application. 

Throughout the CSM program, integration of mathematics and science was 
stressed and activity ideas were presented. CSM researchers also discussed the 
difference between integrated and correlated lessons, but based on teacher reflection and 
interview comments, it appears that the definition of integration was not clarified. As a 
result, although teachers’ awareness of the differentiation between integration and 
correlation increased, their basic conceptions of integration did not change. “Math is the 
language of science” Teacher H said at her May 2010 final interview, and Teacher A 
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“learned that it is a good idea to use other subjects such as science to make [his] lessons 
in math more interesting.” Other teachers espoused the benefits of interdisciplinary or 
thematic teaching on their reflections. Teacher Q felt that “teaching by themes makes it 
easier to integrate different subjects” and Teacher G came to “appreciate that [her 
school’s] Reading [sfc] teachers look at our scope and choose reading material to match 
our science as much as possible.” By the end of the CSM program, at least one teacher 
remained confused as to what exactly counted as an integrated lesson. Teacher I did not 
agree that using mathematics as a tool in science or science as a real world application in 
mathematics counted as integration and was frustrated with her perceived lack of 
creativity. 

Participant teacher incorporation of CSM strategies into classroom 
practices. 

As part of the Satisfaction with Integrated Lessons survey (see Appendix A), 
teachers were asked whether they integrated science and mathematics lessons and the 
frequency with which they taught integrated lessons each week. All eighteen participants 
completed the survey. Fourteen, or 77.8%, responded that they did integrate science and 
mathematics. Of those 14, 78.5% reported that they teach one to two integrated lessons 
per week, 14.3% reported teaching three to four integrated lessons per week, 0% taught 
five integrated lessons per week, and 7.1% did not respond. Assuming teacher self- 
reports are accurate and all other factors are equal, the minimum probability of observing 
an integrated lesson taken at random from the 18 participants is 0. 189. This is calculated 
from (0.2 x 0.778 x 0.786) + (0.6 x 0.778 x 0. 143). 

Participant teachers were observed and interviewed by CSM researchers twice 
during the 2009-2010 AY, once during fall semester, and once during spring semester. 
Researchers took notes during each observation/interview and met afterwards to discuss 
their observations, compile them on the Observation Form (see Appendix C) and rank 
each teacher on how well their lesson was integrated and how well they used effective 
teaching strategies that were advocated during the CSM trainings. A sample of 
completed Observation Forms is provided in Appendix F. Descriptive statistics were 
calculated and charted for the various rankings and qualitative case analysis was done to 




93 



develop a richer understanding of how teaching practices compared between teachers and 
through time as teachers experienced more CSM training. 

CSM researchers evaluated whether the topic taught at each observed lesson was 
appropriate to integrate. If the lesson was appropriate to integrate, researchers then 
determined whether the lesson was taught (a) as an integrated lesson, (b) conceptually, 

(c) using correct language, and (d) using natural and appropriate links between science 
and mathematics concepts. Eleven lessons were observed in the fall semester and the 
incorporation of integration and correlated facets was evaluated for nine of those lessons. 
Sixteen lessons were observed during the spring semester and observation forms were 
available for twelve of those lessons. CSM researchers evaluated all twelve lessons with 
observation forms for their integration appropriateness, but only nine lessons were 
evaluated for incorporation of integration and correlated facets. 

Table 8 depicts a summary of the data describing how many and what percentage 
of observed and evaluated lessons were integrated and incorporated facets of correlated 
lessons. There was a slight decrease in the percentage of lesson topics that were deemed 
appropriate to integrate from fall to spring semester. Also, the percentage of teachers 
who taught integrated lessons when their lesson topic was appropriate to integrate 
decreased from fall to spring semester. It is important to note that spring observations 
were conducted shortly before TAKS testing and many teachers were observed reviewing 
content with their students. Although the sample sizes are small, the percentage of 
teachers who taught integrated lessons who also incorporated facets of correlated lessons 
increased from fall to spring semester. Teachers who integrated lessons in the spring all 
also used each discipline’s language correctly and incorporated natural li nk s between the 
disciplines. No teachers in the fall or spring were observed to teach concepts 
conceptually. 

The percent of observed lessons that were integrated compares favorably against 
the minimum chance (18.9%) of randomly observing an integrated lesson calculated from 
the teacher self-report data gather at the beginning of the CSM program. At the fall 
round of observations, 38.9% (3.5 of 9 lessons) of lessons that researchers observed were 
integrated. During the spring round of observations, 22.2% (2 of 9 lessons) of observed 
lessons were integrated although this percentage may be higher considering the data set is 
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incomplete. Participant teachers were asked to teach integrated lessons on observation 
days if possible and were given advance notice of the observation dates. It is unclear 
whether the percentage of observed integrated lessons was larger than chance because 
teachers made a concerted effort to integrate on observation days or because teachers 
were increasingly applying CSM principles to their teaching practices. 

Table 8 

Descriptive Statistics for the Extent to Which Obserx’ed Lessons Were Integrated and 
Incorporated Facets of Correlated Lessons. 



Observation Form Questions Evaluating 
Lesson Integration & Inclusion of CSM 
Facets 



Was the concept appropriate to integrate? 9 6 67 12 7 58 -8 

If appropriate, was it taught integrated? 6 3.5 58 4* 2 50 -8 

If integrated, was it taught conceptually? 4 0 0 2 0 0 0 

If integrated, was it taught with correct 4 2.5 63 2 2 100 +38 

language? 

If integrated, were discipline links natural? 4 3 75 2 2 100 +25 

*Three of the 7 lessons identified as being appropriate to integrate were not evaluated further. 
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All four of the integrated lessons observed in the fall observation can be 
categorized as science focused with mathematics as a context or tool following the 
Mathematics and Science Continuum (Huntley, 1998; Lonning & DeFranco, 1994). 
Teacher B’s students rotated through five inquiry stations investigating how energy 
affects chemical and physical changes; mathematics was incorporated at one station to 
measure the distances objects travelled. Students in Teacher I’s class experimented with 
dry ice and were asked to find the average time a piece of dry ice takes to sublimate. 
Teacher E led experiments on chemical change and minimally discussed how variables 
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are used in both mathematics and science. Teacher 0, a mathematics teacher, led her 
students through balancing chemical equations by incorporating algebra as a comparison. 
This lesson was closer to the center of the Mathematics and Science Continuum than any 
other observed lesson in the fall or spring. However, Teacher O neither explained 
chemical formula notation well nor clarified how chemical coefficients and subscripts 
correlated to algebraic coefficients and parentheses and observers noted that students 
were confused. 

Both of the integrated lessons observed in the spring can also be categorized as 
science focused with mathematics as a context or tool following the Mathematics and 
Science Continuum (Huntley, 1998; Lonning & DeFranco, 1994). During review of 
science content in preparation for an upcoming Benchmark cumulative test, Teacher K 
used a bar chart to track the performance of each team in a Jeopardy-style review game. 

A portion of the students in Teacher L’s class collaborated to program a robotic dinosaur; 
the programming required that the students understood degrees and reflection, 
mathematics concepts Teacher L reviewed with the students prior to initiation of their 
work. Although some of the teachers incorporated some facets of correlated lessons - 
vocabulary was discipline-appropriate and the li nk s between science and mathematics 
were natural and not contrived - none of the observed lessons were completely correlated 
because all seven facets were not observed and science and mathematics concepts were 
not taught in synergy. 

The effective teaching practices that CSM researchers also ranked teachers’ use of 
during observations included using enhanced content, collaborative learning, effective 
questioning, inquiry, manipulation in the lesson, effective testing, instructional 
technology, enhanced materials, and direct instruction. For each effective practice, 
teachers were graded on a scale of zero to three where zero indicated the practice was not 
observed and three indicated the practice was observed to the greatest extent. 

Additionally, CSM researchers marked a practice as NA, or not applicable, if they 
decided that a practice did not fit within the purview of the lesson. Results for each 
effective practice ranking were plotted on a spider diagram (see Figures 3 - 1 1) to show 
how teachers’ rankings had changed from the fall observation to the spring observation. 
As previously mentioned, many teachers were reviewing material in preparation for the 
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TAKS test at the spring observation. During review, teachers may be less likely to 
incorporate varied teaching practices, and as such may depress the spring teachers’ rating 
scores on effective teaching practices. In order to clarify whether reviewing had a 
depressing effect, spring rankings were plotted twice on the spider diagrams - once 
including all teachers’ rankings, and once excluding the rankings of teachers who were 
reviewing. 



Enhanced context. 




Figure 3. Spider diagram comparing the degree of teachers’ use of enhanced context 
between fall and spring observed lessons. Teachers received scores of 0 - 3 or NA. 
Dataset “Spring All” includes scores from all observed teachers and dataset “Spring NR” 
excludes scores from teachers who were reviewing content for TAKS. 

Teachers at both the fall and spring observations who were not reviewing with 
their students received high scores for the use of enhanced context in their lessons. 

Spring teachers who were not reviewing demonstrated a slightly increased use of 
enhanced context than fall teachers. Teachers that received high scores made their 
lessons relevant to students by using real world examples and activities that were 
interesting to the students. For example, three observed lessons involved sports. Teacher 
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I had students use a pedometer to record their number of steps then graph the results and 
determine the descriptive statistics for the data. Teacher N had her students compete in 
track events, and used their recorded data in a statistics activity. The third teacher (Q) 
used dice rolling and basketball to teach ratios and percentages. Other teachers used 
common household items such as hair-driers, toys and foods in lab activities, taught 
students songs to help remember facts, and encouraged students to discuss connections 
between mathematics and budgeting. 



Collaborative learning. 




Figure 4. Spider diagram comparing the degree of teachers’ use of collaborative learning 
between fall and spring observed lessons. Teachers received scores of 0 - 3 or NA. 
Dataset “Spring All” includes scores from all observed teachers and dataset “Spring NR” 
excludes scores from teachers who were reviewing content for TAKS. 

Employment of collaborative learning during observed lessons was greatest in the 
fall lessons and diminished in the spring lessons. Teachers who were reviewing at the 
spring observation severely depressed the overall spring usage scores. Some reviewing 
teachers had students sitting in groups or teams if they were playing a review game, but 
individuals were not assigned roles, nor did the groups have a specific purpose beyond 
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asking other members to help answer review questions. Teachers who received the 
highest ranking in the fall assigned students to groups to collaborate on a science lab 
inquiry. Those given rankings of two had grouped seating and directed students to work 
an activity as a group, but no group roles were assigned. 



Effective questioning. 




Figure 5. Spider diagram comparing the degree of teachers’ effective questioning 
between fall and spring observed lessons. Teachers received scores of 0 - 3 or NA. 
Dataset “Spring All” includes scores from all observed teachers and dataset “Spring NR” 
excludes scores from teachers who were reviewing content for TAKS. 

Teachers increased their usage of effective questioning strategies from fall to 
spring observations. CSM researchers looked for teachers to vary the timing and 
cognitive level of questions. Most teachers were ranked low on the effective questioning 
scale at the fall observations because choral responses predominated, and they neither 
asked students direct questions nor allowed enough wait time when students replied. By 
the spring observation, some teachers had improved their questioning skills such that 
more teachers were ranked higher in questioning at the spring observation. Teachers who 
received high scores gave students adequate wait time when responding, required 
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explanation of answers, called on non-volunteers, and gave additional explanation as 
needed after students responded. 



Inquiry. 




Figure 6. Spider diagram comparing the degree of teachers’ use of inquiry between fall 
and spring observed lessons. Teachers received scores of 0 - 3 or NA. Dataset “Spring 
All” includes scores from all observed teachers and dataset “Spring NR” excludes scores 
from teachers who were reviewing content for TAKS. 

Inquiry rankings improved somewhat from fall to spring teacher observations. 
Comparing only non-reviewing teachers, roughly the same percentage of teachers were 
graded as using inquiry to the greatest extent at the fall and spring observations, but a 
higher percentage of spring teachers than fall teachers were observed not using inquiry at 
all. The improvement in inquiry usage occurred in the middle of the ranking spectrum as 
teachers moved from using inquiry a little to employing inquiry a moderate amount. 

CSM researchers looked for lessons incorporating student-centered, inductive activities. 
Teachers ranked a one discussed an issue with students without conducting an experiment 
or led students through an experiment step-by-step. Teachers ranked a two had students 
working on an open-ended project such as science fair or a robotics demonstration, but 
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the students seemed a little lost because the teacher didn’t completely understand the 
aspects of inquiry. Regarding the science fair lesson, CSM researchers commented that 
“it was not obvious that the students at this point really knew what they were doing” and 
the “teacher is new to teaching science.” Teachers ranked a three engaged their students 
in inductive labs where students were given general directions and worked in groups to 
discover the results and determine conclusions. Two such activities were science 
experiments and one activity was a Rube Goldberg project incorporating energy 
transformation. 



Manipulation in the lesson. 




Figure 7. Spider diagram comparing the degree of teachers’ use of manipulation between 
fall and spring observed lessons. Teachers received scores of 0 - 3 or NA. Dataset 
“Spring All” includes scores from all observed teachers and dataset “Spring NR” 
excludes scores from teachers who were reviewing content for TAKS. 

The spider diagram for use of manipulatives as a teaching strategy shows that 
teacher practices did not change from fall to spring observations. Teachers either used 
manipulatives or they did not. The primary change at the spring observation was that a 
large number of teachers were reviewing content and researchers determined that 
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manipulative use was not even applicable to the lesson that was observed. All but one 
teacher observed incorporating manipulatives into the lesson were teaching science and 
they employed a wide variety of objects, science equipment and tools during the lesson. 
The mathematics lesson incorporated paper manipulatives. Researchers commented that 
manipulatives could have been employed in the lessons during which they observed no 
manipulative use. Even though several of these classes were reviewing, the researchers 
commented that manipulatives could have helped clarify concepts students were confused 
on. 



Effective testing. 




Figure 8. Spider diagram comparing the degree of teachers’ use of testing between fall 
and spring observed lessons. Teachers received scores of 0 - 3 or NA. Dataset “Spring 
All” includes scores from all observed teachers and dataset “Spring NR” excludes scores 
from teachers who were reviewing content for TAKS. 

The rankings on use of testing suggest that those few teachers observed 
incorporating testing or evaluating their students’ daily learning did so more effectively at 
the spring observation than at the fall observation. No teachers were observed giving 
students a formal or informal end-of-lesson assessment. Instead, teachers were observed 



102 



assessing individual students’ learning as they moved through the lessons by asking 
questions and providing feedback. Teachers receiving a higher ranking asked open- 
ended questions, called on non-volunteers, and provided verbal feedback to students. 
Those receiving a lower ranking asked less cognitively challenging questions, called 
mostly on volunteers and did not provide students with explanatory feedback. The 
majority of observed lessons however were not deemed applicable to incorporation of 
testing. 



Instructional technology. 




Figure 9. Spider diagram comparing the degree of teachers’ use of instructional 
technology between fall and spring observed lessons. Teachers received scores of 0 - 3 
or NA. Dataset “Spring All” includes scores from all observed teachers and dataset 
“Spring NR” excludes scores from teachers who were reviewing content for TAKS. 

A greater percentage of teachers were ranked on the incorporation of instructional 
technology into their lessons at the fall observation than at the spring observation. Use of 
instructional technology was deemed not applicable in about 50% of the spring lessons. 
The pattern of rankings of teachers’ use is similar fall to spring. The important difference 
is that 20% of spring teachers were ranked as incorporating instructional technology to 
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the greatest extent and no teachers were ranked as such at the fall observation. One high- 
ranked teacher, P, incorporated music and songs into review PowerPoint slides which 
some students sang along to and another high-ranked teacher, B, had students working at 
computers to research background information on their science projects. Teachers 
receiving rankings of two used instructional technology such as Smartboards or 
document cameras during a significant portion of the lesson and those receiving rankings 
of one only used instructional technology for a minimal portion of the lesson or with only 
a portion of the students. 



Enhanced material. 




Figure 10. Spider diagram comparing the degree of teachers’ use of enhanced material 
between fall and spring observed lessons. Teachers received scores of 0 - 3 or NA. 
Dataset “Spring All” includes scores from all observed teachers and dataset “Spring NR” 
excludes scores from teachers who were reviewing content for TAKS. 

CSM researchers defined enhanced material as instructional materials modified 
by teachers to better meet the needs of students. Teachers’ use of enhanced material in 
their lessons greatly improved from the fall observation to the spring observation. Even 
teachers who were reviewing at the spring observation incorporated enhanced material. 
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Seventy-five percent of all teachers ranked in the spring were given scores of two or three 
compared to only 13% of teachers ranked in the fall receiving a score of two. The 
reviewing teachers showed creativity in trying to get students to recall and remember 
information. For example, Teacher P created songs to help students remember concepts, 
and several teachers created review packets. Teachers who received rankings of three 
clearly developed much of their observed lesson. Teacher M created an activity 
involving drawing color-coded molecules to clarify chemical formulas and Teacher N 
created a several-day activity incorporating sports into a mathematics lesson on statistics. 



Direct instruction. 




Figure 11. Spider diagram comparing the degree of teachers’ use of direct instruction 
between fall and spring observed lessons. Teachers received scores of 0 - 3 or NA. 
Dataset “Spring All” includes scores from all observed teachers and dataset “Spring NR” 
excludes scores from teachers who were reviewing content for TAKS. 

Direct instruction, according to the CSM researchers, is a best practice when it is 
used to teach skills, techniques, and proper equipment use. Few teachers were scored on 
their application of direct teaching at both observations. Most fall scores fell in the 
moderate use range and related to teachers demonstrating mathematics or science 
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procedures. Spring scores were more evenly spread across the zero to three range, but 
most observed lessons warranted a NA ranking because teachers were reviewing or 
working on a summary lab where new skills were not being learned. There is not enough 
distinguishing data to state whether teachers’ use of direct teaching changed from fall to 
spring. 

CSM Effect on Principals 

The 2009-2010 iteration of CSM included principals in the training and interview 
process in order to increase the coherency of the program and garner administrative 
support for participant teachers as they try to implement new teaching strategies. This is 
the first year principals were included and as such, their feedback and an understanding 
of whether they changed their practices as a result of their participation are important for 
improving CSM for future iterations. 

Participant principal perceptions of the CSM program and strategies. 

Fewer data sets were available for analysis regarding principal participation than 
originally anticipated. Principal perceptions of CSM were estimated from their two days 
of reflections during the summer institute, and interviews from a few principals. 
Appendix G includes principal reflections and Appendix H includes the interview notes. 
Few questions on the summer reflections directly asked for principals’ perceptions of the 
CSM program, but the two most involved principals did make their views clear. Most 
principals did comment in reflections or during interviews on concepts they learned new 
concepts from CSM. 

For the purposes of analysis, principals were grouped by their attendance records 
as “committed” to, “interested” in or “disinterested” in the CSM program. Two 
principals were categorized as committed because they attended all of the summer 
institute days and at least two Saturday sessions. Three principals who attended the 
summer institute but no Saturday sessions were categorized as interested. One principal 
attended only one Saturday session, but this training was held at her school, and another 
principal did not attend any CSM training sessions. These two principals were 
categorized as disinterested. 
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Principal perceptions of CSM. 

The committed principals enjoyed the CSM program and found it valuable. Both 
noticed that after participating in CSM, teachers collaborated more often, and brought 
back teaching ideas they shared with other teachers. Principal X also noted that her 
teachers gained science and mathematics content knowledge. Principal T was very 
involved in the CSM training and sat in on some of the teacher Saturday sessions. She 
was pleased enough with the CSM program that she decided to add more teacher teams 
from more grade levels for the next year. She wanted greater participation by her science 
and mathematics teachers and desired greater vertical integration of her staff. Principal T 
also commented that she might run her own control group of teachers at her school to 
better understand the impact of participation in CSM professional development. 

Three principals suggested areas where the CSM program could improve or better 
meet the needs of their specific schools. Principal T wanted to experience more hands-on 
activities and desired more time to attend and observe the teacher portion of CSM. 
Principal X wished there was a physical language take-away for her teachers, requested 
that sample lessons taught by participant teachers during CSM training be videotaped, 
and requested that strategies for classroom management be addressed in the teacher 
sessions. Principal Y requested that mathematics content focus on algebraic reasoning, 
measurement, and problem solving (specifically TAKS objectives 2, 4, and 6); at the time 
of this principal’s comment, these topics were already planned for the 2010-2011 CSM 
agenda. 

What principals learned during CSM. 

All committed and interested principals commented on what they had learned 
from the CSM program. All these principals expressed gaining a better understanding of 
what inquiry is and how important it is, especially to science teaching. Some participant 
principals better understand the problems confusing and similar language pose, how to 
use student performance data to guide teachers and better target instruction to student 
needs, and how to focus teacher observation walk-throughs to look for certain teaching 
strategies. Additionally, Principal T, who had a mathematics background, commented 
that she gained understanding of science content and science teaching best practices such 
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that she felt more comfortable evaluating science teachers and had a better idea of what 
effective science teaching looked like. 

Effect of CSM on principals’ management of science and mathematics 
teachers. 

Several instruments were analyzed to assess whether and how participant 
principals altered their management of science and mathematics teachers after 
participating in the CSM professional development program. Reflection questions (see 
Appendix D) from the summer institute asked principals to consider how they would 
integrate CSM strategies into their management practices. Notes from principal and 
teacher interviews were analyzed to identify what actions principals had taken since 
participation in CSM and what actions could still be taken to further support participant 
teachers. Results on how principals’ management styles have changed after participating 
in CSM are presented below in case-study fonnat, ordered by attendance categories. 

Committed principals. 

Principal T. 

Principal T was the most active principal in the CSM program and her writings 
and interviews indicate that she valued the program and did try to incorporate what she 
learned into her work. She wrote in her summer reflections that she planned to use 
student performance data to “plan areas of focus with staff, work the plan, assess along 
the way, [and] evaluate,” advocate integration with her teachers, and change her style of 
teacher observation by focusing on only one or two areas of interest per observation. 
Principal T reported at the last Saturday CSM session that she had changed her teacher 
observation practice. She felt better able to evaluate science teacher effectiveness 
because of the science content and pedagogy best practices she learned during CSM, and 
she now asks students in the class she is observing what the students are learning. 

Principal T’s participant teachers identified obstacles that, if Principal T had 
addressed, may have better allowed the teachers to incorporate CSM strategies into their 
teaching practices. The teachers wished for more time to collaborate and complained that 
C-Scope, their school’s adopted scope and sequence, did not align with CSM best 
practices such as teaching one concept per lesson. Additionally, Principal T picked two 
participant teachers for the CSM program who were not on the same teaching team, did 
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not share a conference period, and only shared about 20% of their students. This teacher 
pair, as a result, was not able to easily collaborate during the school year. It might have 
been better for Principal T to pick participant teachers who had a greater chance of 
working together during the school year. 

Principal X. 

Principal X reflected at the summer institute that she planned to promote 5E at her 
school, allocate weekly and monthly time for teacher collaboration, and use a rubric 
during her teacher observations to help her identify hands-on activities and student- 
directed teaching. CSM researchers observed that Principal X did allocate time for 
teacher collaboration, but it is unclear if the amount of allocated time changed after 
Principal X participated in the CSM program. Principal X did actively support the 
collaboration of her participant teachers. The teachers were encouraged to discuss their 
ideas, and at the spring CSM teacher observation, the teachers were observed teaching 
together. 

Interested principals. 

Principal U. 

During the summer institute, Principal U wrote that he hoped to “incorporate 
inquiry into all subject areas,” use a modified DuFour model to promote teacher 
collaboration, and prioritize integration such that it “drive[s] our PBL instruction.” No 
evidence was gathered to indicate whether any of these plans were initiated or whether 
Principal U’s management style changed after attending CSM training. 

Teacher interview comments suggested Principal U was supportive of their 
attendance at CSM and subsequent ideas. For example, Principal U provided his teachers 
with substitute teachers so they could attend CAST and so the 6 th grade participant 
teachers could meet for a half-day during school to discuss ideas. Additionally, he agreed 
to let one participant science teacher change the order of science topics from the 
mandated scope and sequence to allow students time to learn the required concept 
(proportions) in mathematics class prior to having to apply it in science class. 

Participant teachers of Principal U discussed several problems which hindered 
their implementation of CSM teaching strategies that Principal U may have been able to 
mitigate, but had not as of the time of the interviews. Teacher R did not have a 
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complementary science partner at CSM training and would have liked to collaborate with 
one. She also indicated that the mandated mathematics scope and sequence was too rigid 
to readily allow incorporation of CSM teaching strategies. Several of Principal U’s 
teachers complained that they did not have enough planning time with their own 
department colleagues, let alone with their complementary discipline’s colleagues. It is 
difficult for teachers to integrate science and mathematics when they cannot meet as a 
group to plan. 

Principal Z. 

Principal Z apparently came away with fewer concrete plans than Principal U 
even though she attended the entire summer principal institute and Principal U only 
attended half. Principal Z did write that she would “use the domains to focus more on 
whether and to what degree 5E [is] being implemented,” meaning that she planned to 
look for evidence that teachers are employing inquiry in their classrooms during 
classroom walk-through observations. Principal Z indicated general support for CSM 
strategies but was vague in how she would specifically do this only writing that she 
would “give [teachers] every opportunity to leam about and implement science and math 
integration.” 

No evidence was gathered to demonstrate that Principal Z altered her observation 
methods or increased promotion of mathematics and science integration. Limited 
evidence demonstrated that Principal Z supported the teaching strategies of CSM. 

During a CSM observation of Teacher G, Principal Z demonstrated understanding of the 
importance of avoiding confusing language with students by reinforcing the CSM 
observers’ opinions that Teacher G needed to be clearer about the definition and usage of 
the word “prime”. According to participant teachers, Principal Z could have better 
supported them by organizing vertical team meetings for the teachers to work on scope 
and sequence, integration ideas, consistent vocabulary, and other topics. 

Disinterested principals. 

Principal Y. 

Principal Y did not attend any portion of the CSM principal training but her 
participant teachers were the most motivated and collaborative pair in implementing 
CSM strategies and integration ideas. Prior to attending CSM training, Principal Y’s 
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teaching staff already used a communication tool called Curriculum on the Wall which 
allowed them to see what topics were planned for each discipline throughout a time 
period. The CSM participant teachers reported that Curriculum on the Wall served as a 
good foundation which helped them move into integrating science and mathematics. 

No data were gathered which indicate that Principal Y provided support for her 
participant teachers beyond allowing them to attend the CSM program. Instead, the 
participant teachers’ enthusiasm and ideas for implementing CSM strategies were not 
capitalized upon. Teachers O and P spent time collaborating and expressed desire to 
team teach science and mathematics. They were willing to give up conference periods 
and teach in different rooms, but Principal Y apparently did nothing to advance this. 
Additionally, Teachers O and P were interested in working in vertically aligned teams 
beyond the grade levels in their school, but received no support to schedule such a 
meeting. The teachers indicated that the mandated scope and sequence, C-Scope, did not 
align with CSM best practices such as one concept per lesson, and while they were 
currently able to work in some CSM strategies and integrated lessons, C-Scope was 
required for the next school year and the teachers feared they would not be able to 
continue working in integration lessons. 

Summary. 

In summary, although principals who attended the summer CSM institute 
recorded plans and ideas to incorporate into their management practices, little data was 
available to detennine the degree to which principals acted on those plans or whether the 
management style of the principals after attending CSM training was much different than 
before training. Minimal data suggested that the principals categorized as committed did 
change two management habits. Principal T started talking to students during walk- 
through observations, and Principal X followed through on her plan to allocate time for 
teacher collaboration. It is uncertain if these principals or any other principals made 
other changes to their management practices after attending the CSM program. 

Principals categorized as committed and interested did support their teachers’ 
participation in the CSM program generally and to some degree supported ideas the 
teachers brought back from the CSM program. Principals demonstrated their general 
support by attending CSM trainings and promoting their teachers’ attendance. Principal 
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X allocated collaboration time for teachers, Principal U granted a waiver for one teacher 
to alter the scope and sequence order, and Principal Z reinforced a teacher’s use of proper 
discipline language, a facet of the CSM program. 

Participant teachers identified common barriers to implementation of CSM 
teaching strategies that participant principals could have acted upon during the 2009- 
2010 AY. Several teachers attended CSM without a complementary partner, and in one 
instance the partner they did attend with was not on the same teaching team at their 
school. Teachers of most principals desired more time to collaborate with their 
complementary partner, share with their colleagues, or participate in meetings with 
teachers of several grade levels in order to vertically align curricula. Finally, teachers 
complained that their school-adopted scope and sequence was not flexible enough to 
allow them to try lessons using CSM strategies, and several teachers whose scope and 
sequence was C-Scope complained that C-Scope lessons did not align with the one 
concept per lesson best practice promoted by the CSM training. 

CSM Effect on Students 

A primary goal of professional development is to increase the effectiveness of 
educators such that student learning is enhanced (Loucks-Horsley et ah, 2010). Student 
achievement is partially influenced by teacher knowledge and skills (Guskey & Sparks, 
1996). Although it can be difficult to correlate specific student achievements with 
professional development (Guskey & Yoon, 2009), when such data are available, they 
reinforce teachers’ attitudes towards strategies obtained from professional development 
(Guskey, 2002b). 

Student performance on TAKS. 

Participant and control teachers were asked to submit their aggregate TAKS 
scores for the 2008-2009 and 2009-2010 school years. No control teachers submitted 
TAKS scores. Therefore, research question 6a which asks how the perfonnance of 
students of participant teachers, as measured by the percentage of students who met 
Texas standards on the science and mathematics TAKS examinations, compares to the 
performance of students of non-participant teachers, cannot be answered. Five 
participant teachers submitted TAKS scores for both 2009 and 2010. Two of those 
teachers, however, changed either grade level or subject taught and therefore their 2009 
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and 2010 TAKS scores were not compared as there is a clear confounding variable. 
Table 9 shows the 2009 and 2010 TAKS scores of students of three participant teachers 
who taught the same grade and subject both years. 

Table 9 

TAKS Results of Students of Participant Teachers: School Years 2008-2009 and 2009- 
2010 



Teacher 




2009 TAKS 




2010 TAKS 




No. 

Students 

Tested 


% Students 
Met 

Standards 


% Students 
Commended 


No. 

Students 

Tested 


% Students 
Met 

Standards 


% Students 
Commended 


G 


128 


80 


35 


98 


81 


20 


O 


69 


83 


26 


66 


83 


23 


P 


151 


68 


21 


163 


66 


23 



For each teacher, G, O, and P, a one-tailed binomial test was completed to 
determine whether the percentage of students who met TAKS standards in 2010 was 
significantly higher or lower than the percentage of students who met TAKS standards in 
2009, each test’s hypothesized value. The 2010 percentage of students who met TAKS 
standards was not significantly greater than the 2009 percentage of students who met 
TAKS standards for teacher G (one tailed p = .500), or teacher O (one-tailed p = .55 1), 
and the 2010 percentage of students who met TAKS standards was not significantly 
lower than the 2009 percentage for teacher P (one-tailed p = .285). 

Similarly, for each teacher, G, O, and P, a one-tailed binomial test was conducted 
to determine whether the percentage of students who earned commended TAKS scores in 
2010 was significantly higher or lower than the percentage of students who earned 
commended TAKS scores in 2009. The 2010 percentage of students who were 
commended on TAKS was not significantly higher than the 2009 percentage of students 
who were commended for teacher P (one-tailed p = .326) and was not significantly lower 
for teacher O (one-tailed p = .327). However, the 2010 percentage of students who were 
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commended was significantly less than the 2009 percentage of students who were 
commended for teacher G; one-tailed p = .001 with a negative effect size of 0.15. 

Results of student pretests and posttests. 

Students of participant and control teachers were administered physics or 
mathematics (depending on the discipline of the teacher whose class students were in) 
content pretests and identical posttests at the beginning and end of the 2009-2010 school 
year, respectively. One-way ANCOVA was carried out for each content test, physics and 
mathematics, to determine whether there was a significant difference in the degree of 
achievement between students of participant and non-participant teachers. The 
independent variable, teacher type, had two levels - teachers who participated in CSM 
and teachers who did not participate in CSM. The dependent variable was the student 
score on the content posttest and the covariate was the student score on the content 
pretest. 

For the physics content test, an analysis of the homogeneity-of-slopes assumption 
showed no significant difference in the relationship between the posttest score and the 
pretest score as a function of teacher type, F( 1,545) = 2.196, MSE = 228.66, p = .139, 
partial rf = .004. The physics content test ANCOVA was not significant, F(1.546) = 
2.003, MSE = 229. 16, p = .158. There was no significant difference in the degree of 
student achievement in science between students of participant and non-participant 
teachers as measured by physics content pretests and posttests. 

For the mathematics content test, an analysis of the homogeneity-of-slopes 
assumption also showed no significant difference in the relationship between the posttest 
score and the pretest score as a function of teacher type, F( 1,536) = .123, MSE = 278.72, 
p = .726, partial rf = .000. The mathematics content test ANCOVA was significant, 

F( 1.537) = 20.568, MSE = 278.28, p = .000, partial rf = .037. There was a significant 
difference in the degree of student achievement in mathematics between students of 
participant and non-participant teachers as measured by mathematics content pretests and 
posttests, such that students of participant teachers outperformed students of non- 
participant teachers. The small effect size indicated by the partial rf suggests the 
relationship between teacher type and student posttest score is weak to moderate. 
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Although not addressed in the original research questions, ANCOVA models 
were attempted to detennine whether student posttest scores were significantly different 
for students with more participant teachers for mathematics and science class than 
students with no participant teachers. The independent variable, number of participant 
teachers, had three levels - zero, one, or two. The dependent variable was the student 
score on the content posttest and the covariate was the student score on the content 
pretest. An analysis of the homogeneity-of-slopes assumption for the physics content test 
showed a significant difference in the relationship between the posttest score and the 
pretest score as a function of number of participant science and mathematics teachers, 
F(2.543) = 3.448, MSE = 225.10, p = .033, partial rfi = .013. The analysis of the 
homogeneity-of-slopes assumption for the mathematics content test also showed a 
significant difference in the relationship between the posttest score and the pretest score 
as a function of number of participant teachers, F(2,534) = 4.532, MSE = 277. 14, p = 

.011, partial rf = .017. Therefore, ANCOVA was not conducted for the physics and 
mathematics content tests because results would not be meaningful (Green & Salkind, 
2005). 
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Chapter V 
Discussion 

Guskey’s Evaluation Model 

To evaluate the effectiveness of the 2009-2010 implementation of the CSM 
professional development, the results will be discussed within the framework provided by 
Guskey’s (2000) five-level model for evaluating professional development. Level one 
considers how participants reacted to the program, level two evaluates what and how well 
participants learned new concepts or skills, level three assesses how CSM worked with 
and affected organizational support and change, level four evaluates how well 
participants applied their new knowledge in the classroom or principal’s office, and level 
five considers whether CSM had an impact on student achievement. 

Level one - participant’s reaction to CSM. 

Participant perceptions of the CSM program provide important feedback to help 
improve the design and execution of future rounds of CSM professional development 
(Guskey, 2000). Based on largely positive responses from participant teachers and 
committed principal participants, CSM was effectively planned and facilitated. Overall, 
teacher participants were pleased with the 2009-2010 CSM program and found the 
content and activity ideas interesting and relevant to their teaching practices. Teachers 
were very satisfied with the predominance of student-centered, hands-on activities and 
discussions that were incorporated into the training; this is similar to teacher perceptions 
Duran (2010) reported for CSM cohort four. The focus on active learning kept teachers 
engaged and challenged throughout the training. When teachers were able to meet a 
challenge, their self-assurance was bolstered. The focus on effective teaching strategies 
for science and mathematics during CSM gave some teachers feelings of validation 
regarding teaching practices they already employed. Additionally, teachers commented 
that they appreciated having time during the professional development to discuss 
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concepts with their colleagues as the discussions sparked new ideas and enabled teachers 
to learn both the CSM content and other relevant teaching strategies from one another. 

All three classes, CMRS, CPM, and PBL, were enjoyed by the participants. The 
CMRS class received the most positive reactions from participant teachers. Several 
teacher reflections did compliment the CMRS instructor’s teaching style. Teachers did 
come to CSM with a higher level of mathematics background knowledge than physics as 
evidenced by their college transcripts and their pretest scores on the mathematical 
reasoning test which were much higher than the pretest scores of the physics test. 
Additionally, as evidenced by their learning reflections, more teachers readily picked up 
activity ideas and teaching strategies from the CMRS class than from the CPM class. 
Teachers may have preferred the CMRS class because they were better prepared to get 
more out of it than they were from the CPM class. That being said, teachers also 
thoroughly enjoyed the PBL class which they found exciting because they got to create 
Rube Goldberg devices and apply robotics, activities which most participants had little 
experience with. The teachers left the PBL class with positive collaborative group 
experiences and an excitement for incorporating real-world applications into their 
teaching practices. 

Based on the principal attendance data, I infer that the majority of participant 
principals did not fully commit to the CSM program and did not see their continued 
attendance through the AY as valuable or a good use of their time. Committed principals 
however, those who did attend the majority of CSM principal training sessions, did enjoy 
the professional development program and found it valuable. Principals expressed desire 
to spend more time during the training on hands-on activities and participating alongside 
their participant teacher teams. While the CSM professional development succeeded in 
incorporating much active learning and collective participation at the teacher level, it fell 
short of these important effective professional development aspects at the principal level. 

Suggestions for improvement. 

Although participant teachers’ experiences with CSM were positive overall, some 
teacher responses and reactions to specific events suggest areas where the professional 
development program can be improved. Regarding structural aspects of CSM, teachers 
wished for more time during the CMRS class for independent practice with new concepts 
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and manipulative tools, and more time during PBL class to work on their group projects. 
Some teachers would have liked to cover more content and other teachers were interested 
in receiving extension materials to take away. The 2009-2010 CSM program included a 
summer institute that met for ten days over the course of two weeks and five AY 
Saturday sessions. Perhaps participant teachers might better benefit if the program were 
longer to incorporate more independent practice and group work time or more content, or 
at least more spread out to allow more time for teachers to assimilate new knowledge into 
their existing schemas. The summer institute could meet three or four days a week over 
the course of three weeks or AY Saturday sessions could meet once a month. 

Participant teachers grew weary of completing thrice-daily reflections during the 
summer institute. Teachers also commented that the reflection questions given following 
the PBL class did not align well with what they were engaged in during class. A separate 
set of reflection questions should be developed to better fit the PBL activities. 
Additionally, reflections questions could be worded differently to garner more truly 
reflective responses from participants. For example, reflection questions three (what did 
you learn that you didn’t know before) and four (what did you leam that added depth or 
breadth to previous knowledge) are very similar and teachers tended to either give the 
same response with slight wording differences to both questions or skip question four 
altogether. 

Many participant teachers were not pleased with the facilitation of discussions 
during CPM class. The discussions were too long, involved too few participants, and the 
content being discussed was at too high a level for some teachers to understand. This led 
5 th and 6 th grade teachers to feel excluded and uncomfortable. Many teachers, especially 
teachers of lower grades and those with lower levels of background knowledge, became 
confused during and frustrated with the CPM discussions. In future iterations of the 
CPM class, CSM instructors should ensure the topics for discussion are within most 
participant teachers’ zone of proximal development, plan specific goals for the 
discussions and guide the discussion toward such goals, and encourage more discussion 
participation by teachers. 

The summer institute was dense with content and teaching strategy ideas. 

Teachers of lower grades and those with lower levels of background knowledge 
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expressed feeling overwhelmed and found that a lot of material was new to them. When 
teachers experienced content above their knowledge level, they were more likely to 
become confused and/or frustrated. Additionally, 5 th and 6 th grade teachers lost self- 
confidence and 5 th grade teachers experienced anxiety with not being able to keep up. 
When teachers were frustrated, they developed empathy for their students (5 th and 6 th 
grade teachers) or each other (8 th grade teachers), which may be a positive side effect of a 
negative incident. Given that the CSM program targets middle school teachers and there 
is a large differential in depth and breadth of science and mathematics content between 
5 th and 8 th grade, it is no surprise that participant teachers of lower grades struggled with 
the CSM content more than participant teachers of higher grades. In order to minimize 
confusion, frustration, and feelings of inadequacy among 5 th and 6 th grade teachers during 
future iterations of CSM, CSM researchers might consider several options. First, the 
CSM researchers might purposefully group teachers in different ways during the training 
depending on the activity. It may make sense to group teachers of higher grades with 
teachers of lower grades (perhaps groups of four with 8 th grade teacher partners paired 
with 6 th grade teacher partners and 7 th grade teacher partners paired with 5 th grade teacher 
partners) in order to allow the teachers of higher grades to tutor or support the teachers of 
lower grades during content work. At other times, perhaps during consideration of level- 
specific pedagogy, it may make more sense to group teachers of lower grades together 
and teachers of higher grades together. Second, if the teacher content pretests were 
completed at a session much earlier than the bulk of the professional development, 
perhaps a week ahead of time, CSM instructors could use the pretest data to better align 
the planned content to the knowledge levels of the in-coming participants. Third, 
instructors might consider giving participants preread material prior to attending such that 
when participants arrive for the first day of training, they are starting with some similar 
ideas and levels of knowledge. 

Level two - participants’ learning. 

Evaluation at level two considers what and how well participants learned new 
content and skills or altered their attitude or beliefs after completing the CSM 
professional development program (Guskey, 2000). The 2009-2010 CSM program was 
highly effective in increasing participants knowledge of content and content pedagogy. It 
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is clear from both the quantitative and qualitative data that participant teachers learned 
significant amounts of mathematics and physics content and picked up a number of new 
activity ideas and teaching strategies. 

As demonstrated by teachers’ scores on the content pretests and posttests, the 
CSM professional development program significantly improved participant teachers’ 
knowledge of mathematical reasoning and physics. The effect size on score 
improvement in mathematical reasoning was medium (d = 0.746) and the effect size on 
score improvement in physics was large (d = 1.21 1). Teachers demonstrated a much 
larger gain in physics content knowledge than they did in mathematical reasoning. This 
is likely due to participant teachers’ general lack of prior physics knowledge as compared 
to their prior mathematical reasoning knowledge. The average score on the physics 
pretest was much lower than the average score on the mathematical reasoning pretest 
(45.76 versus 73.50, respectively) and therefore participant teachers had more room to 
demonstrate growth on the physics posttest. Recall that only four participant teachers had 
taken any physics classes in college. Additionally, throughout their reflections, teachers 
commented that they had little experience with physics content and that many of the 
topics covered in the CPM class were new to them. These results and the differential in 
score gains between physics and mathematics are similar to what were found by Gloyna 
(2008) who also suggested that teachers’ higher levels of mathematics background 
knowledge may have caused higher test scores. 

The difference in prior knowledge between physics and mathematical reasoning 
impacted what teachers learned during CSM classes. Teachers acquired basic content 
knowledge in CPM class, but in CMRS class they learned more in-depth content, to 
better understand the why of something, and acquired more teaching strategies. Many 
CPM reflections commented on learning definitions and fonnulas such as distance, 
displacement, velocity, speed, open systems, closed systems, and types of energy. 
Physics-specific activities and teaching strategies were only commented on occasionally. 
Reflections for CMRS however were much more diverse in nature. Teachers did 
comment on learning basic content, but they also commented largely on understanding 
why a concept is taught a certain way (i.e. fraction division or unit conversion) and 
learning new ways to teach concepts. Manipulatives were incorporated into the CMRS 
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class to a large extent. Manipulatives served two purposes; they helped teachers increase 
their content knowledge and also served as models for teaching mathematics concepts 
which teachers could easily assimilate in their own practices. CPM class included hands- 
on aspects, but not to the degree that CMRS incorporated manipulatives. It is possible 
that the more hands-on nature of the CMRS class also affected whether participant 
teachers learned more than just content. Many teachers commented in their reflections 
that they thought they learned better via hands-on activities. 

As a result of attending the CSM professional development, participant teachers 
developed an awareness and understanding of the problem that discipline-specific 
language can cause students. Most teachers were not aware that mathematics and science 
use similar language and symbols in different contexts and that the vocabulary can have 
different meanings in different disciplines. Identifying confusing language and parallel 
ideas and using language properly within a discipline are key facets of correlated lessons 
(West & Browning, 2010). 

Suggestions for improvement. 

The CSM professional development program is strong in affecting participant 
learning. In order for teachers to become effective integrators, they require a sound 
understanding of the content in both disciplines they wish to integrate (Berlin, 1994; 
Frykholm & Glasson, 2005; Furner & Kumar, 2007; Huntley, 1999). Although 
participant teachers made significant gains in both physics and mathematical reasoning 
content knowledge, their level of physics understanding was less upon departure from 
CSM than that for mathematical reasoning because teachers started the program with a 
low level of physics knowledge. This disparity in understanding between physics and 
mathematical reasoning may prevent teachers from integrating the two as much as they 
would like. Gloyna (2008) reported that teachers in CSM cohorts one through three 
continued to feel inadequate with content outside of their primary discipline after 
attending CSM training. CSM researchers might consider offering two sections of 
physics content to try to bridge the science-mathematics level gap. A week-long 
intensive course in basic physics could be offered to participant teachers prior to 
attending the full-fledged CSM professional development. The CPM class offered during 
the regular session could then be geared towards a more intermediate knowledge level 
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such that teachers would acquire more physics teaching strategies on par with what they 
garnered from the 2009-2010 CMRS class. 

The mathematical reasoning pretest and posttest may be too easy for the 
participant teachers. On the pretest, four teachers scored 92% or above, none scored 
below 52%, and the average score was 73.5%. Such high scores leave little room for 
teachers to demonstrate their full growth. Additionally, if so many participants score so 
well on the pretest which follows the class itinerary, they likely would benefit from an 
expanded scope even though very few teachers expressed boredom during CMRS class. 

Participant teachers did not change their conceptions of integration during CSM 
except to learn about correlated lessons and how they differ from integrated lessons. The 
preconceptions of integration that teachers brought to CSM span the diverse range of 
ideas held by educators in general. Participant teachers felt that science and mathematics 
complemented each other well and tools and applications from one discipline can be used 
in the other to increase the relevancy of a lesson to students. Teachers framed integration 
in terms of thematic or interdisciplinary teaching. A clear definition of integration 
apparently was not stressed by CSM instructors, but neither is there a single definition of 
integration currently accepted among researchers (Berlin, 1994; Berlin & Lee, 2005; 
Berlin & White, 1992; Hurley, 2001). A few teachers were still confused at the end of 
the CSM program as to what integrated lessons really entail, and therefore CSM 
instructors might work to clarify and better convey their understanding of integration to 
participant teachers in future iterations of CSM professional development. 

Level three - degree of organizational support & change. 

Level three evaluation detennines the degree to which a professional development 
garnered administration support and affected organizational change (Guskey, 2000). The 
CSM professional development program made a concerted effort to establish 
administration support and spark organizational change. The 2009-2010 CSM program 
was revised from prior implementations to include a parallel training for principals of 
participant teachers and AY interviews with the participant principals to check their 
status and assess their needs. CSM researchers offered support to principals throughout 
the AY and were responsive to principal requests. Despite the efforts of CSM 
researchers, participant principals overall were not completely committed to the program 
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and did not offer significant support to their teachers so the 2009-2010 CSM program 
was only minimally effective in affecting organizational change. 

CSM did successfully raise principal awareness of the importance of inquiry in 
science and mathematics instruction, and the difficulties discipline-specific language can 
cause students. Principals also received a better understanding of what effective science 
and mathematics teaching looks like and how to evaluate such teaching practices during 
principal walk-throughs. 

Overall principal support for the program was varied however as evidenced by 
their attendance records and collected responses. Most principals attended the two days 
of CSM training during the summer institute, but very few continued to attend CSM 
sessions during the AY. This lack of consistent attendance by principals implies a lack of 
complete support for CSM. Principals did support full participation by their teachers, in 
some cases granting teacher groups extra time during school to collaborate, and 
advocated CSM implementation by providing their school buildings for use during AY 
weekends, but most principals did not prioritize their own continued participation and 
learning. 

Organizational change therefore was limited mostly to what participant teachers 
could accomplish on their own and this varied widely based on degree of teacher buy-in 
and initiative. Following the findings of Johnson (2006, 2009) teachers face political, 
technical, and cultural barriers to implementing effective teaching strategies and those 
teachers who face the fewest technical and cultural barriers can find ways to circumvent 
political barriers. Two extreme cases from the 2009-2010 CSM implementation illustrate 
Johnson’s findings well. Case one includes the science and mathematics teacher team 
(Teacher P and Teacher O, respectively) from school D. This teacher team’s principal, 

Y, was a disinterested principal and offered little support to the team or relevant comment 
on the program during interview. Teachers O and P however, showed the most initiative 
and collaborated more than any other teacher participants. The team showed interest in 
the CSM concepts and discussed ideas for their school throughout the CSM training, they 
made a concerted effort to plan lessons together during the AY, and planned a team- 
teaching day in their cafeteria. Case 2 includes Teachers K and A from school F. The 
principal of this team, T, was the most committed participant principal of the CSM 
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program, but the teacher team was the least interested in the CSM program. Teacher K 
was observed sleeping in class and neither teacher collaborated well during the institute. 
Neither made any effort during the AY to collaborate with each other. They did not share 
many of the same students at school F and were actually on separate teaching teams, but 
neither is there evidence that they shared their knowledge or tried to collaborate with 
complement discipline teachers within their own teaching teams. Principal T saw value 
in the CSM program and was excited enough by its potential to plan to send more teacher 
teams from more grade levels for the 2010-2011 CSM program. In both cases, teachers 
received the same training to minimize technical barriers and faced similar political 
barriers such as lack of coincident planning time and resources. The team in case one 
also received no administrative support, but had low cultural barriers (they saw value in 
integration and the ideas promoted by CSM) and were able to work around some political 
barriers. The team in case two did receive administrative support, but did not work 
together and had high cultural barriers, and therefore was not able to overcome their 
political barriers. 

Suggestions for improvement. 

For future iterations of the CSM principal programs, CSM researchers should 
consider whether the correct administrators participated. Several participants were from 
large schools. In a large school, the principal may not directly oversee specific 
departments; an assistant principal may manage certain academic departments or grade 
levels or minimally, the principal may defer to a department head. For large participant 
schools, CSM researchers should investigate whether the principal, assistant principal, or 
department head is the appropriate administrator to invite. 

Additionally, the monetary incentive offered to participant principals apparently 
was not adequate to secure principal attendance beyond the summer institute. CSM 
researchers should survey principals to determine what would get them to prioritize CSM 
attendance and revise the principal incentive to match their requirements. It is possible 
principals might be motivated by something besides money. 

It is not clear whether specific barriers to integration that teachers face were 
discussed in the principal trainings. Principals were given ideas for supporting 
integrative teaching such as implementing Project Learning Committees, but there is no 
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evidence from the principle reflections or interview data that principals became aware of 
the many technical, political, and cultural barriers that teachers face when implementing 
changes to their teaching practices (Johnson, 2006, 2007). Participant teachers identified 
several political barriers that their principals could have worked to minimize had they 
been aware of them. Teachers desired more collaboration time during the school day and 
requested vertical alignment meetings. Some required additional resources to implement 
CSM activities. Teachers noted that the school adopted scope and sequences were not 
flexible enough to allow time to experiment with CSM strategies and some curriculum 
guides did not align with the ideals of CSM. Additionally, not all participants attended 
with a complementary partner. It would be worthwhile for the principal sessions of 
future CSM programs to discuss these and other barriers to implementation that teachers 
face. Such information might induce better principal attendance and attention to these 
problems back at their individual schools. 

Level four - participants’ application of acquired knowledge or skills. 

Level four evaluation examines the degree to which participants applied the 
knowledge or skills they acquired during the CSM professional development to their 
practice (Guskey, 2000); the 2009-2010 CSM program was moderately effective at level 
four. The CSM program provided participants with a content-rich program which 
focused on effective science and mathematics teaching strategies for integration. 

Teachers were observed in their classrooms twice during the AY and CSM researchers 
rated them on nine measures of effective teaching and several measures of their level of 
integration and use of correlated lesson facets. Evaluation of the degree of change in 
teacher practice was complicated by the timing of the second teacher observation which 
fell during TAKS review time and limited several teachers’ use of integration and 
effective teaching practices. 

It was clear from the creativity of observed lessons that some teachers made a 
pointed effort to integrate on observation days, as was requested by CSM researchers. 

The number of teachers reviewing at the second observation however depressed the 
numbers of integrated lessons seen at the spring observation. Fewer lesson topics were 
found to be appropriate to integrate at the spring observations than at the fall observations 
and the percentage of lessons appropriate to integrate that were taught with integration 
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also fell from the fall to spring observations. Gloyna (2008) reported that participant 
teachers of CSM cohorts one to three taught integrated lessons on observation days the 
year they were required to do so and Duran (2010) reported that 78% of participant 
teachers in CSM cohort four taught integrated lessons on observation days. These results 
are much better than what was observed with cohort five, the 2009-2010 set of 
participants. Duran reported that cohort four teachers were only observed once during 
the AY and it was not clear whether the observations were perfonned in the fall or spring 
semester or whether they were affected by TAKS timing. It is not clear why so few 
cohort five teachers were observed integrating on observation days. 

The rate at which cohort five teachers incorporated facets of correlation however, 
increased from the fall to spring observations suggesting that teachers were becoming 
more comfortable with the correlation facets and incorporating them into their lessons 
more frequently. Teachers certainly became aware of the language disparities between 
disciplines during CSM and had made an effort to be more accurate and specific with 
their language while teaching. At the last CSM AY session, when asked how they have 
altered their teaching practice, several teachers remarked that they were more cognizant 
of their language use and had worked to be more precise with their students. The 
awareness of and change in practice regarding language use is different than what was 
reported by Gloyna (2008) and Duran (2010). Gloyna reported that although cohort one 
to three teachers claimed to understand the problems caused by context-specific 
language, vocabulary was used incorrectly during lessons that CSM researchers observed. 
Similarly, Duran reported that cohort four teachers were sloppy with their language use 
during sample lessons presented at the summer institute. The improved understanding of 
and proper use of language by cohort five teachers indicates that the CSM program has 
been improved such that the language facets are more effectively conveyed and multiple 
teacher observations allow for the discovery of more changes to teacher practice 

Most of the observed integrated lessons were minimally integrated and 
categorized as science focus with mathematics as a context or tool (Huntley, 1998; 
Lonning & DeFranco, 1994). This aligns with the teachers’ lack of change in 
conceptions on integration as discussed earlier. While science focus with math does fall 
on the Mathematics and Science Continuum (Huntley, 1998; Lonning & DeFranco, 
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1994), CSM researchers focused on correlated lessons, which maximally integrate 
science and mathematics. Failure of teachers to demonstrate correlated lessons during 
observation days suggests that teachers need more time to work on integration and more 
experience with examples of integration models. 

Participant teachers did demonstrate increased usage of the six of the nine 
measured effective teaching strategies promoted during the CSM program. As compared 
to the fall observations, teachers demonstrated increased application of enhanced context, 
effective questioning strategies, inquiry, testing, instructional technology, and enhanced 
materials during spring observations. The degree of use of manipulatives and direct 
teaching was similar during the fall and spring observations. The only teaching strategy 
that lost ground from fall to spring was the use of collaborative learning, but this may 
have been depressed by the amount of TAKS reviewing being done at spring 
observations. 

Suggestions for improvement. 

Although existing data indicates that participant teachers did incorporate their 
new knowledge into their teaching practices to a moderate degree, there are aspects of the 
CSM program that could be changed to strengthen this inference. First, spring 
observations during future CSM programs should be timed such as not to coincide with 
TAKS review because reviewing teachers did not integrate lessons or incorporate 
effective teaching practices as much as non-reviewing teachers did. It is difficult to get a 
clear picture of how teachers’ teaching practices are changing from fall to spring if one 
lesson is review and therefore not completely comparable to the other observed lesson. 
Second, trends in teacher practice would be more strongly identified if there were more 
than two observations per teacher. Additionally, observing teachers prior to their 
participation in CSM would provide researchers with a baseline from which practices 
observed after participation can be compared against. Third, if CSM researchers also 
observed lessons of control teachers at the same times and intervals of participant 
teachers, a second baseline and comparison would be available to detennine whether 
changes in participant teacher practices during the AY were clearly attributable to CSM 
participation or whether some changes might be attributed to other professional 
development trainings or school-wide initiatives all teachers experienced during the AY. 




127 



Additionally, CSM researchers could request that both participant and control 
teachers complete daily or weekly logs of practices they have applied, or submit 
portfolios of lesson plans taught throughout the year. Logs or portfolios would increase 
the amount of data upon which an evaluation of association of teaching practice with 
CSM participation could be made. Logs completed on observation days would serve as 
anchor points such that teacher-reported data from non-observation days could be 
nonnalized to researcher evaluations. 

Finally, use of the effective teaching practice checklist CSM observers used to 
rank participant teachers could stand improvement. The dataset was incomplete for the 
2009-2010 CSM program; not all teachers were ranked on all categories. Additionally, a 
rubric might be created to specify what each level, zero, one, two, or three, of an effective 
teaching practice looks like. The length of time between fall and spring observations is 
large and the lack of a predetermined rubric can lead to more subjective rankings. For 
example, the rankings for inquiry did not seem to be consistent; it was difficult for this 
evaluator to differentiate why a lesson was ranked a two or a three in inquiry. 

Level five - impact on student achievement. 

Evaluation of CSM at Level 5 assesses whether CSM indirectly affected student 
achievement (Guskey, 2000). The 2009-2010 CSM program was moderately effective in 
improving student achievement. CSM researchers planned to collect two types of data on 
student achievement - TAKS scores and student scores on science and mathematics 
content pretests and posttests. The collection of student TAKS scores was not successful 
and therefore a comparison of the performance of students on the TAKS test was not able 
to be made between students of participant and control teachers. Of the TAKS data that 
was collected from participant teachers, only score sets from three participant teachers 
were able to be analyzed. Significance tests were conducted to compare, for each 
teacher, the percentage of students who met TAKS standards and the percentage of 
students who earned commended scores between 2009 and 2010. Only one comparison 
was found significant; the percentage of students of Teacher G who earned commended 
scores in 2010 was significant lower than the percentage of students who earned 
commended scores in 2009. 
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Overall, this set of results from TAKS score comparisons does not clarify whether 
CSM professional development affected student achievement because there are numerous 
confounding variables. First, the students teachers taught during AY 2009 were not the 
same students teachers taught in AY 2010 and there was no way to control for student 
differences. Second, Teacher G, who showed a significant decrease in the percentage of 
her students who earned commended TAKS scores from AY 2009 to AY 2010, was 
teaching in a different environment in AY 2010 than in AY 2009. In AY 2009, she 
taught in a self-contained 5 th grade environment, but in AY 2010, her school switched to 
a departmentalized 5 th grade environment. Third, teachers experience different stressors, 
are accountable for implementing different school initiatives, and participate in different 
professional development programs from year to year which could not be controlled for 
with the limited data available here. 

In the future, CSM researchers could take several steps to increase the usefulness 
of collected TAKS scores. The effects of CSM on student achievement as measured by 
TAKS scores might be better indicated by trend analysis because of the difficulty in 
controlling for the effects of different student bodies, teacher stressors, school initiatives, 
and attendance at other professional development programs. CSM researchers could 
collect several years of TAKS scores from participant and control teachers to see whether 
there is a change in slope after participation in CSM professional development. By 
including scores from control teachers, CSM researchers might be able to partially 
control for school initiatives and school-wide professional development trainings and 
might also be able to compare results between participant schools. Duran (2010) 
suggested similar improvements as she also was not able to produce meaningful results 
from two years of TAKS scores. 

CSM researchers did succeed in administering and collecting student achievement 
data from the science and mathematics pretests and posttests from students of both 
participant and control teachers. Students of participant teachers demonstrated 
significantly larger achievement gains in mathematics than students of control teachers as 
measured by content pretests and posttests. A weak to moderate relationship was found 
between teacher type and student mathematics posttest score. There was no significant 
difference in student achievement gains in science as measured by the science content 
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pretests and posttests between students of participant and control teachers. The results of 
the student pretest and posttest comparisons suggest that the CSM professional 
development program did indirectly improve student achievement in mathematics. 

It is important to note however, that participant teachers and control teachers were 
not randomly chosen or assigned. Participant teachers self-selected or were chosen by 
their principals for attendance at CSM. Control teachers were recruited by principals. It 
is unknown if the skill-levels of participant and control teachers were equitable. 
Additionally, there were not equal numbers of control teachers as participant teachers and 
not all participant schools contributed control teachers to the study. Only 1 1 control 
teachers were recruited, but there were 18 participant teachers. School E is a specialized 
community of teachers focused on mathematics, science, and technology excellence. 
School E did not contribute any control teachers, but six of the 18 participant teachers 
were from School E. The achievement of students of participant teachers could be biased 
in the favor of participant teachers because of this. Also, no control teachers taught 5 th 
grade and so the students of participant 5 th grade teachers could have pulled the average 
achievement scores down because the content pretests and posttests included material 
from grades 5-8, most of which 5 th grade students have not learned. 

It is difficult in real life school settings to develop and complete a truly 
experimental study of a professional development. That being said, CSM researchers 
could try to obtain a more balanced set of participant and control teachers in future 
iterations of the program. Ideally, the same number of participant and control teachers 
teaching the same combination of grade levels and disciplines and representing the same 
schools would better balance the experimental comparisons. 

Summary of CSM effectiveness. 

The 2009-2010 implementation of the CSM professional development program 
was effective based on positive assessments in four of the five levels of Guskey’s (2000) 
evaluation framework. Participants had a positive reaction to most aspects of the CSM 
program. Participant teachers enjoyed the active learning that dominated the classes and 
found the content relevant to their current needs. Committed principals found the 
program valuable. Participant teachers clearly improved their content knowledge of 
mathematics and physics as their scores improved significantly from both content pretests 
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to the posttests and they reflected much on the new content they were learning. 

Participant teachers also developed an awareness of the problem discipline-specific 
language can cause students and developed their skills for using manipulatives and other 
hands-on activities in the classroom. Participant teachers did try to integrate in their 
lessons after the CSM summer institute and observed integrated lessons were dominantly 
classified as science focus with mathematics. Participants increased their application of 
facets of correlated lessons as the AY progressed; teachers especially were paying more 
attention to their use and teaching of language within each discipline’s context. Teachers 
also increased or maintained their use of eight of the nine effective teaching strategies 
stressed in CSM and evaluated by CSM observers during AY lesson observations. The 
CSM professional development is associated with improved student learning. Students of 
participant teachers demonstrated significantly larger gains in mathematics achievement 
as measured by a mathematics pretest and posttest than students of control teachers. 

The 2009-2010 CSM program was not as effective in garnering organizational 
support and promoting change. The CSM program was specifically designed to include 
principals this iteration; principals were offered principal-specific CSM training, follow- 
up support and resources. However, principals were not fully committed, did not take 
advantage of all offered to them, and only provided minimal support to their participant 
teachers. Limited organizational change was affected only by a few motivated 
participant teachers who were able to overcome barriers without aid from principals. 

Summary of suggested program changes. 

The 2009-2010 CSM professional development program does have room for 
improvement in its structure, class instruction, and experimental design. Structurally, the 
summer institute might be extended to or spread out across a third week to allow 
participants more time to consider and study new concepts and teaching strategies and 
minimize participants’ feelings of overwhehnedness. In order to boost teachers’ content 
background knowledge to common levels in preparation for the CSM institute, teachers 
might be given preread material or offered a more basic level physics or mathematics 
class prior to the regular CSM institute. Content pretests and posttests could be revised to 
ensure the range of question difficulties is broad enough to capture knowledge gains of 
all participant teachers. Teacher reflection sessions might be streamlined and reflection 
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questions for the PBL class should be modified to better align with the PBL activities. 

AY observations should be better timed, especially during spring semester, to avoid 
major periods of review in preparation for district- wide and state- wide tests. A third or 
additional observation would provide better understanding of how well participant 
teachers are applying skills learned at the summer institute 

The CSM program was strong in participant instruction and general facilitation of 
the classes and AY sessions. A few modifications might be considered. During the CSM 
summer institute, participant teachers might be dynamically grouped to encourage peer 
tutoring or grade-level discussion throughout the classes. Second, facilitation of class 
discussions should be improved such that all participant teachers are involved and the 
content level of the discussion stays within the realm of comprehension for most 
participants. 

Participant principals apparently require a different incentive to participate fully 
than the monetary incentive they received during the 2009-2010 iteration. Although 
CSM is effective in increasing teacher knowledge and practices, and is associated with 
improved student achievement in mathematics, principal buy-in is necessary for teachers 
to continue implementing CSM strategies and conveying best practices to colleagues. 

The reach of the CSM professional development program over the long tenn will be 
limited if principals do not increase their active support of the program. 

If further investigations into the effectiveness of CSM are planned then several 
aspects of experimental design could be improved. First, more effort should be made to 
enroll participant teachers who are more representative of their school’s teacher 
populations and an equivalent number and quality of control teachers from the same 
schools who teach the grades 5-8 and mathematics and science in similar proportions as 
the participant teachers. Second, evaluation of teacher lessons needs more rigor. All 
teachers should be assessed in all categories of integration and effective practices. The 
Best Practice Rubric within the Teacher Observation Form (see Appendix C) should be 
expanded to specify what teacher behaviors or practices equate to rankings of zero to 
three; this would increase objectivity and comparability of the Fall and Spring 
observation notes. A third or additional observation might be added to the evaluation 
plan to increase the data on teacher application of CSM skills; an observation made prior 
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to teacher attendance at the CSM summer institute would serve as a baseline data point - 
a pretest of teacher practices. Third, if comparison of TAKS scores continues in the 
evaluation plan for assessment of student achievement, then more effort is required in the 
collection of such data. TAKS scores need to be collected from all participant teachers 
and control teachers for a range of years prior to and including the AY of the CSM 
program and subjected to trend analysis. 

Conclusion 

The Correlated Science and Mathematics professional development program is an 
effective model of professional development. The design of CSM meets accepted 
standards for effective professional development, the 2009-2010 implementation of CSM 
met the goals of the CSM program and evaluation of the 2009-2010 implementation 
following Guskey’s (2000) five-level framework indicate the program was effective at 
four of five levels and minimally effective at one level. 

Standards for effective professional development. 

The 2009-2010 CSM program invited pairs of mathematics and science teachers 
to participant with their principals in an intensive summer institute and follow-up AY 
sessions. The program focused on science and mathematics content, pedagogy, and 
integration teaching strategies for teachers, and focused on clarifying and effectively 
evaluating integration, inquiry and effective science and mathematics teaching strategies 
for principals. The design of CSM fits into the framework for effective development 
(Birman et ah, 2000) in that it is mostly a traditional fonnat, spans an entire school year, 
and strongly urges participants to attend collectively, as mathematics-science partners 
from the same school along with their principal. Additionally, CSM is focused heavily 
on content, employs active learning, and maintains communication with participants and 
their schools to ensure participants’ needs are being addressed. CSM is also standards- 
based and considerate of participants’ beliefs, both additional characteristics of 
professional development (Guskey, 1991; Guskey & Yoon, 2009; Johnson, 2009; 
Loucks-Horsley et al., 2010). 

Research questions. 

Five of the seven proposed research questions were able to be answered with the 
available data. As hypothesized for research question one, participant teachers did 
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express positive perceptions of the CSM professional development program. They 
appreciated the focus on active learning, content, and relevant teaching strategies. As 
hypothesized for research question two, participant teacher content knowledge of both 
physics and mathematics, as measured by pretests and posttests, did significantly improve 
as a result of attendance at the CSM summer institute. Participant teachers additionally 
commented on learning how to teach with manipulatives, why science and mathematics 
language can be confusing to students and how to mitigate such confusion. As 
hypothesized for research question three, qualitative data did provide examples of 
teachers incorporating integration and inquiry into their teaching practices. Observations 
of practicing participant teachers identified that teachers made some effort to integrate 
mathematics and science lessons such that lessons classified as science focus with 
mathematics were observed. Teachers increased their usage of effective teaching 
practices stressed during CSM such as using enhanced content, effective questioning 
techniques, inquiry, testing, instructional technology, and enhanced materials. 
Additionally, teachers developed an awareness of the importance of proper language use 
within each discipline and began to alter their teaching practices to ensure they used 
language correctly per context and prevent confusion among their students. 

Research questions four and five concerned the effect of CSM on participant 
principals. The hypothesis for question four, that principals will express positive 
perceptions of CSM was partially correct. Participant principals enjoyed the CSM 
summer institute, but few principals continued their participation into the AY. 

Committed principals, those who attended most of the CSM program, felt CSM was 
valuable to them. Research question five which asked how principals’ management of 
science and mathematics teachers was affected by CSM, was not able to be answered 
well with the available data. Principals only demonstrated minimal support of participant 
teachers’ trying new strategies and although principals did increase their understanding of 
inquiry and effective science and mathematics teaching strategies, it is unclear from the 
data available whether principals changed their management practices to reflect this. 

Research questions six and seven concerned the indirect effect CSM had on 
students of participant teachers. Question six which asked whether CSM indirectly 
influenced student achievement on TAKS was not able to be answered with the available 
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data. The hypothesis for research question seven, that the students of participant teachers 
will achieve significantly better than students of control teachers in science and 
mathematics as measured by pretests and posttests was half correct. Students of 
participant teachers did achieve significantly better than students of control teachers in 
mathematics, but the student achievement in science was not significantly different. 

CSM goals. 

The study results indicate that the 2009-2010 implementation of CSM did meet its 
program goals. Goal 1 was to increase teacher content knowledge; the program clearly 
met this goal. Goal 2 was to improve best practices in teaching science and mathematics. 
This goal was met as CSM observers found participant teachers increased and improved 
their use of enhanced context, effective questioning strategies, inquiry, testing, 
instructional technology, and enhanced materials and maintained their use of 
manipulatives and direct teaching. Goal 3 was to increase integration of science and 
mathematics concepts in the classroom. CSM observers found some teachers integrating 
at the science focus with mathematics classification. Teachers demonstrated increased 
usage of correlated facets, especially those regarding proper language, throughout the 
AY. Goal 4, to improve student performance in science and mathematics was partially 
met. As compared to students of control teachers, students of participant teachers 
demonstrated significantly greater achievement in mathematics as measured by the 
pretest and posttest. However, students of participant teachers did not achieve 
significantly differently than students of control teachers in science as measured by the 
science pretest and posttest. Goal 5, to improve principal knowledge of and abilities to 
evaluate inquiry in science and mathematics lessons, was also partially met. Principals 
did increase their awareness and understanding of inquiry and what effective inquiry 
looks like. At least one principal changed their teacher evaluation habit, but not enough 
data was available to determine whether other principals also did so. 

Guskey’s framework. 

Evaluation of the study results within the framework of Guskey’s (2000) model 
for evaluation of professional development indicates that the 2009-2010 implementation 
of the CSM program was effective. The CSM program was very effective at Guskey 
levels one and two - providing participants with a positive, quality experience as well as 
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increasing participants’ knowledge and skills. The CSM program made strong efforts to 
gamer organizational support, Guskey level three, but these efforts were minimally 
effective. Principals supported the program but not enough to minimize barriers for 
teachers trying to implement CSM teaching strategies. CSM was moderately effective at 
affecting change in participants’ teaching practice, Guskey level four. Teachers 
demonstrated increased use of facets of correlated lessons and effective teaching 
practices championed by CSM. The CSM program was also moderately effective at 
Guskey level five, affecting student outcomes. CSM is associated with increasing student 
achievement in mathematics. No association can be made regarding student achievement 
in science. 

The CSM professional development program is effective at training teachers in 
science and mathematics content, pedagogy, and integration teaching practices. 

Principals, who commit to the program, develop a broader understanding of inquiry, 
integration, and effective science and mathematics teaching practices, and available data 
suggest committed principals likely alter their management practices to incorporate their 
new knowledge. Students of participant teachers demonstrate increased achievement in 
mathematics. The results from the 2009-2010 implementation of CSM show 
improvements over those of prior evaluated cohorts and continue a positive trend of 
teacher, principal, and student growth. 
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Satisfaction with Integrated Science and Math Lessons 
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1) A. Do you integrated science and math in your lessons? If no, skip to question 2. 

a) Yes 5 b) No 

B. How many integrated lessons do you currently do each week? 

a) 1-2 b) 3-4 c) 5 

C. Where do you get the integrated lessons? Place a check mark in the appropriate 
column (Never, Sometimes, Frequently). For each resource that you use, rate your 
satisfaction level on a scale of 1-5, with J_ being not satisfied and 5 very satisfied , 
in the Satisfaction Level column. 





Never 


Sometimes 


Frequently 


Satisfaction 

Level 


Yourself 










Integrating 
Teacher (math or 
science) 










Online 










Other Teachers 
(not math or 
science) 










Other Resources 











D. What are the problems you have encountered with integrating science and math? 
Circle the appropriate number on a scale of 1-5, with 1 being a major problem and 
5 being no problem . 



a) Time 1 

b) Coordination of students 1 

c) Planning for instruction as a team 1 

d) Coordination of student assessments 1 

e) Availability of models and/or materials 1 

f) Communication 1 

g) Exposure to integration in past 1 

h) Content knowledge in science 1 

i) Content knowledge in math 1 

i) Training 1 

j) Strict scope and sequence 1 



2 3 4 5 
2 3 4 5 
2 3 4 5 
2 3 4 5 
2 3 4 5 
2 3 4 5 
2 3 4 5 
2 3 4 5 
2 3 4 5 
2 3 4 5 
2 3 4 5 
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E. Please list any additional problems you have encountered with integrating that 
were not listed above. 



2) What reasons would you have for having students participate in integrated learning? 



3) What do you think is the value of integrated learning in mathematics/science? 
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Appendix B 

Teacher Daily Reflection Form 

Mix It Up 
Daily Reflections 

NAME: COURSE: Physics & Math □ 

Mathematical Reasoning & Science I I 

Project Based Learning O 

Journal your impressions: 

What was your favorite part of today’s lessons? What did you like about it? 

What was your least favorite part of today’s lessons? What did you dislike about it? 

What did you leam that you didn’t know before? 

What did you leam that added depth or breadth to previous knowledge? 

What other topics or concepts or TEKS came to mind during the lesson? 
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Appendix C 

Observation and Interview Summary Form 

TQ “Mix It Up”; Correlated Science and Math 
OBSERVATION FORM 



Teacher 


Grade Level 


Class Size Room Size 


District 


School 




Principal 


Date of Observation 


Lesson Topic 


TQ Topic 



Teacher’s Schedule: 



Part I: 

1. Was the concept appropriate to correlation? 

2. If so, was the lesson correlated? 

a. Was each discipline taught conceptually? 

Recommendations: 

b. Was the Language of each discipline correct? 
Recommendations: 

c. Was the link between the science and math appropriate/natural? 



Recommendations: 
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3. Best Practice Checklist/Rubric: Best Practice instructional strategies are measured 
on a 3 -point scale ranging from being observed 0 (not at all) to 3 (greatest extent). 



Effective Strategies 


1-3 


Comments 


Enhanced Context (real world, science fair, problem/case based, 
use tech, to bring in real world, relating learning to students' 
previous experiences, knowledge or interests, Problem Based 
Learning, field trips, use schoolyard for lessons, encouraging 
reflection, hurricanes, global warming) 






Collaborative Learning (arrange students in flexible groups w/ 
assigned roles to work on various tasks, e.g. conducting lab/field 
activities, inquiry projects, group science fair projects, discussion, 
heterogeneous.) 






Questioning (varying time, positioning, or cognitive levels of 
questions, e.g. increasing wait time, adding pauses at key 
student-response points, including more high-cognitive-level 
questions, stopping visual media at key points and asking 
questions) 






Inquiry (student-centered, inductive instructional activities, e.g. 
using guided or facilitated inquiry activities, guided discoveries, 
inductive lab activities, indirect instruction. Using Descriptive, 
Comparative or Experimental designs.) 






Manipulation (opportunities to work or practice with physical 
objects, e.g. operating apparatus, developing skills using 
manipulatives, drawing or constructing something) 






Testing (changes in frequency, purpose, or cognitive levels or 
evaluation, e.g. providing immediate or explanatory feedback, 
using diagnostic testing, formative testing, retesting, testing to 
master) 






Instructional Technology (use tech, to enhance instruction, e.g. 
using computers, etc. for simulations, modeling abstract 
concepts, and collecting data, showing videos to emphasize a 
concept, using pictures, photographs, or diagrams, wikis, pod 
casts, blogs) 






Enhanced Material (modified instructional materials, e.g. 
rewriting or annotating text materials, tape recording directions, 
simplifying lab apparatus to meet student needs) 






Direct Instruction (teach skills, how to use equipment, 
techniques, etc.) 
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Part II: 

4. Interview with Teacher 



A. Typical science lesson? 




B. Was the same lesson taught previously? 




C. What are some examples of either CSM or integrated 
lessons you used in your classroom? 




D. How well did it/they work? How did you measure its 
effectiveness? Will you teach it again? How would you 
change it to teach again? 




E. If none, then why? 




F. How else can we help you either correlate or integrate 
science & math? 




G. What did or could your Principal do to help? 




H. What, if any, changes this year in your classroom or test 
scores (individual classroom tests or district 
benchmarks) or any area can you discern? 




1. How many CSM or integrated lessons have you &/or 
your team done this year? 




J. Which of those would you attribute to the Mix It Up 
training? 




K. What changes to the science and/or math program have 
occurred this year? 




L. Which of those would you attribute to the Mix It Up 
training? 




M. What were your overall goals with this teacher this year 
regarding math or science instruction? 




N. What goals did you accomplish this year concerning 
math or science instruction? 




0. Of those goals, describe which are ones that you feel 
were most influential in student success in math or 
science learning? 




P. What evidence of student learning in science or math do 
you have for this cohort of students (eg how well your 
students did in 7 th grade last year compared to 8 th grade 
this year)? 




Q. How has student academic performance increased for 
this class (eg. Last year 8 th grade compared to this year 
8 th grade)? 




R. To what extent do the disaggregated data show a 
reduction in achievement gaps in math or science? 




S. What was your biggest challenge in instruction for math 
and/or science this year? 




T. What was your greatest tool in overcoming that 
challenge? 




U. What percent of your school budget is for science 
laboratory equipment for last year and for this year? 
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V. What percent of your school budget is for science 
laboratory supplies for last year and for this year? 




W. What percent of your school budget is for math 
equipment for last year and for this year? 




X. What percent of your school budget is for math supplies 
for last year and for this year? 




Y. What was the most valuable part of the program for 
you? 




Z. How do you think the program can be improved? 





Part III: 

5. Interview with Principal 



A. What were your overall goals with this teacher this year 
regarding math or science instruction? 




B. What, if any, changes this year can you discern in this 
teacher's classroom instruction? 




C. What, if any, changes this year can you discern in this 
teacher's curriculum? 




D. What, if any, changes this year can you discern in this 
teacher's room appearance? 




E. What, if any, changes this year can you discern in this 
teacher's collaboration between team members? 




F. What, if any, changes this year can you discern in this 
teacher's training of other teachers? 




G. What goals did you and this teacher accomplish this year 
concerning math or science instruction? 




H. Of those goals, describe which are ones that you feel were 
most influential in student success in math or science 
learning? 




1. Has the teacher made comment to you about their 

thoughts on the program or implementing the program in 
class? 




J. What, if any, changes this year between team members can 
you discern? 




K. What, if any, changes this year in collaboration among 
sci./math departments or other departments can you 
discern? 




L. Which of those would you attribute to the Mix It Up 
training? 




M. What changes to the science and math program have 
occurred this year? 




N. Which of those would you attribute to the Mix It Up 
training? 




0. What evidence of student learning in science or math do 
you have for this cohort of students (eg 7 th grade last year 
compared to 8 th grade this year)? 
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P. How has student academic performance increased for this 
class (eg. Last year 5 th grade compared to this year 5 th 
grade)? 




Q. To what extent do the disaggregated data show a reduction 
in achievement gaps in math or science? 




R. What changes or structures did you add to the school 
schedule this year to promote collaboration among the 
math and sciences (extra planning period, common 
planning periods, early release days, weekly after school 
professional learning communities, etc...) ? 




S. How frequently do your teachers engage in data discussions 
in math and/or science? 




T. Did you model inquiry in any of your staff development or 
staff in-service training? 




U. How have you encouraged consistent and intensive 
vocabulary development in math and science? 




V. What evidence do you have of increased teacher adherence 
to Best-Practices and research-based pedagogy in math and 
science? 




W. How have you encouraged teachers in the Mix-lt-Up 
program to share their skills with other math and science 
teachers as well as with teachers outside of math and 
science? 




X. What percent of your staff/professional development 
budget was spent on science training? 




Y. What percent of your staff/professional development 
budget was spent on math training? 




Z. What correlation do you see with teacher performance and 
attendance at math or science staff/professional 
development? 




AA. What activities have you implemented to improve student 
and teacher attitudes about science? 




BB. What activities have you implemented to improve student 
and teacher attitudes about Math? 




CC. What percent of your school budget is for science 
laboratory equipment for last year and for this year? 




DD. What percent of your school budget is for science 
laboratory supplies for last year and for this year? 




EE. What percent of your school budget is for math equipment 
for last year and for this year? 




FF. What percent of your school budget is for math supplies for 
last year and for this year? 




GG. What would you like to see in the program? 
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Part IV: 

Met Com Met Com Met Com Met Com 
Districts TAKS: 



Science: ‘06 
Districts TAKS: 


‘07 


‘08 


’09 


Math: ‘06 

School TAKS: 


‘07 


‘08 


’09 


Science: ‘06 
School TAKS: 


‘07 


‘08 


’09 


Math: ‘06 

Teacher TAKS: 


‘07 


‘08 


’09 


Science: ‘06 
Teacher TAKS: 


‘07 


‘08 


’09 


Math: ‘06 


‘07 


‘08 


’09 



District Benchmarks 




Appendix D 

Principal Daily Reflection Form 
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NAME: 

Journal your impressions: 

Day 1 

How can you use teacher and student science and math perfonnance data to improve 
instruction? 



How will you facilitate science and math integration in your school? 



What student engagement strategies can also be useful in other content areas? 



Day 2 

What changes in your teacher observations can you make to improve science and math 
integration? 



What is your action plan to help your math and science teachers collaborate? 



How has your view of inquiry been expanded? 



How will you know if the Mix-It Up program has been successful for you and your 
teachers? 
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Appendix E 

Sample of Teacher Reflections 



Sample CMRS Reflections: 

NAM E : Teacher A 7/22 COURSE: Physics & Math I I 

Mathematical Reasoning & Science 1X1 
Project Based Learning O 

Journal your impressions: 

What was your favorite part of today’s lessons? What did you like about it? 

I really enjoyed the opportunity to practice several times the concepts after the initial 
exploration or learning portion of the lesson. I think sometime we avoid allowing time to 
practice because we are afraid to use "worksheets". 

What was your least favorite part of today’s lessons? What did you dislike about it? 

I thought the entire lesson was very informative. 

What did you leam that you didn’t know before? 

No new material. 

What did you leam that added depth or breadth to previous knowledge? 

The lesson today reinforced new ways to teach students about fractions using pattern 
blocks. Some of the methods used today, I can directly transfer to my classroom. 

What other topics or concepts or TEKS came to mind during the lesson? 

I was reminded of topics other than fractions that I can use some of the things used today, 
(proportions etc) 
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NAM E : Teacher E 7/29 COURSE: Physics & Math I I 

Mathematical Reasoning & Science 1X1 
Project Based Learning O 

Journal your impressions: 

What was your favorite part of today’s lessons? What did you like about it? 

Once again using the Cuisenaire rods to learn the concepts in a new way. I love getting 
to see it conceptually and understand things that I learned years ago by memorizing but 
never really seeing the process. 

What was your least favorite part of today’s lessons? What did you dislike about it? 

For me it is actually harder sometimes to see all the different ways to do one problem 
because when I actually understand one way all the others confuse me. I understand why 
it is important to show that though. 

What did you leam that you didn’t know before? 

How to use the Cuisenaire rods to do division 

What did you leam that added depth or breadth to previous knowledge? 

Stated above 

What other topics or concepts or TEKS came to mind during the lesson? 

How can I apply this to my classroom in science? 
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Sample CPM Reflections: 

NAM E : Teacher I 7/21 COURSE: Physics & Math 1X1 

Mathematical Reasoning & Science I I 
Project Based Learning O 

Journal your impressions: 

What was your favorite part of today’s lessons? What did you like about it? 

I enjoyed flicking the penny game. . . Why because it was fun. 

What was your least favorite part of today’s lessons? What did you dislike about it? 

I did not enjoy the questions after questions that were never ending. I think the 
discussion went on WAY TO LONG. I understand the point was to make us think. . . 
however for me it was very frustrating. We discussed what a reference point was for 
over an hour and then finally the answer we all HAD BEEN SAYING... was 
determined. . . and then that was it. 

Another thing was the way everyone was talking on and on and on about nothing. We 
couldn’t hear everyone, and one person was totally controlling the discussion. 

I hope this class will not be like this daily. . . it was very frustrating for me. 

What did you leam that you didn’t know before? 

I have a new perspective on reference point and point of reference. 

I also learned the correct vocab for x and y : ) 

What did you leam that added depth or breadth to previous knowledge? 

The whole discussion today was added MUCH DEPTH maybe too much to my 
understandings Distance and Displacement, and Point of Reference. 

What other topics or concepts or TEKS came to mind during the lesson? 

How frustrated students can become and totally shut down. . . I experienced it! ! ! 
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NAM E : Teacher B 7/22 COURSE: Physics & Math 1X1 

Mathematical Reasoning & Science I I 
Project Based Learning O 

Journal your impressions: 

What was your favorite part of today’s lessons? What did you like about it? 

I needed to use the dry erase board to trace out the bug’s journey and I enjoyed drawing 
and doing calculations. 

What was your least favorite part of today’s lessons? What did you dislike about it? 

I almost dreaded the conversation portion but found I could “keep up” with the ideas 
flying around. Perhaps I am feeling more comfortable? 

What did you leam that you didn’t kn ow before? 

I valued the discussion of what is and is not a correlated lesson but I realize that such a 
conversation would be meaningless with me earlier. 

What did you leam that added depth or breadth to previous knowledge? 

I at times can be sloppy with my language and am guilty of assuming that students 
comprehend subjects only because of their ability to use vocabulary. 

What other topics or concepts or TEKS came to mind during the lesson? 

I am at a blank. 
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Sample PBL Reflections: 

NAM E : Teacher L 7/21 COURSE: Physics & Math I I 

Mathematical Reasoning & Science I I 
Project Based Learning 1X1 

Journal your impressions: 

What was your favorite part of today’s lessons? What did you like about it? 

The explore portion of Lego building. I liked the playing of the parts and just seeing what 
it does. 

What was your least favorite part of today’s lessons? What did you dislike about it? 

The sharing portion of the exploring because the person I was paired up with went way to 
fast and was just clicking here and there. I did like that the proff. Addressed that now 
your part do the play part. Human dynamics of sharing and finding the right partners. 

What did you leam that you didn’t kn ow before? 

I learned about Lego and Veneir websites and now I can go explore at my leisure. 

What did you leam that added depth or breadth to previous knowledge? 

I began to think of ways to integrate technology other than my promethean activ board 
but the challenge becomes $$$$$ from the district to buy even 1 set of the Lego Mindset. 

I also learned what else the scientific calculator can do. 

What other topics or concepts or TEKS came to mind during the lesson? 

Intergration of technology, application of design and redesign, writing up what you 
leamed(ela strands), listening/speaking strands, extended lessons to home, getting 
students excited about design and engineering at the 5 th grade level. Building connections 
to math and science through technology... using my masters degree finally! ! ! Yahoo. 
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NAM E : Teacher N 7/28 COURSE: Physics & Math I I 

Mathematical Reasoning & Science I I 
Project Based Learning |XI 

Journal your impressions: 

What was your favorite part of today’s lessons? What did you like about it? 

I enjoyed seeing all the ideas for the machines and how people implemented them. Even 
though some projects used similar components they were all vastly different. 

What was your least favorite part of today’s lessons? What did you dislike about it? 
Nothing comes to mind 

What did you leam that you didn’t kn ow before? 



What did you leam that added depth or breadth to previous knowledge? 

I could see how everyday problems could be used to illustrate science. It seemed like a 
great way to teach kids about energy transfers and simple machines because it was really 
concrete and relevant. 

What other topics or concepts or TEKS came to mind during the lesson? 
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Appendix F 

Sample Completed Teacher Observation Forms 



Sample 1, Fall 2009: 

TQ “Mix It Up”; Correlated Science and Math 
OBSERVATION FORM 

Teacher O Grade Level _8 Class Size 15 Room Size 

District _D School D_ Principal Y_ 

Date of Observation Oct. 15 Lesson Topic rational & irrational numbers problems 
TQ Topic 

Teacher’s Schedule: 

2:03 S distributes notebooks for wann-up (5-7) for Ss to complete while T checks role. S 
asks for help w/ #8. T - T teaches S how to solve #6 w/ some questions. S - what is 
variable, T - letter. T writes on board while S calls out numbers correctly. 

149. 68+. 48+9 1.3 = ? Warm-up not link to lesson. How link to balancing equations or 
?? 

T uses PPt w/ balances an equation C02 + H20 = H2C03. Ss are counting atoms. S 
asks why there is no “1” after the “C” T- balanced cause same number of atoms on each 
side. Slide 2 Ag+ S = Ag2S. Relate how link back to rational/irrational #s. 

REACTANT + REACTANT = PRODUCT. 

Slide 5 Propane & oxygen wields carbon dioxide and water. Coefficient 502 = 5x=10. 
All call outs. Used language like coefficient and subscripts. T ”we haven’t used 
subscripts in math yet.” No questioning of non-volunteers 

2:22 - T passes out bag w/ 6 green cards & 5 yellow cards. Ss distributing white boards 
& markers. S says markers give her a headache. (S04)3 T had to put 3 in front 3(S04) 
for S to understand to multiply. For problems w/ 2HC1, T asks if need a parenthesis 
cause in math, need parenthesis. Perhaps use only problems that are balancing equations. 
NEED TO SLOW INPUT/MODEL SO STUDENTS CAN LEARN THE PATTERN 
BEFORE THE PRACTICE. Students trying to balance equations adding subscripts 
instead of coefficients. SS SHOULD NOT TRY TO BALANCE EQUATIONS YET, 
ONLY IDENTIFY IF BALANCED OR NOT. 

Source lab? X-Treem Science modification (which had too many examples). T - when 
you take TAKS, what will you remember? Atoms count. 

1. Was the concept appropriate to integrate? Yes 

2. If so, was the lesson integrated? Yes 
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a. Was each discipline taught conceptually? No 
Recommendations: Lesson was on rational numbers and balancing 
equations. Teacher also discussed distributive property. Teacher did 
not relate distribute property to equation correctly. 

b. Was the Language of each discipline correct? No 
Recommendations: Teacher did not explain balancing equations 
correctly. Perhaps more discussion with science teacher. 

c. Was the link between the science and math appropriate/natural? Yes 
Recommendations: 

3. Best Practice checklist/Rubric: Best Practice instructional strategies are measured on 
a 3-point scale ranging from being observed 0 (not at all) to 3 (greatest extent). 



Effective Strategies 


0-3 


Comments 


Enhanced Context (real world, science fair, problem/case 
based, use tech, to bring in real world, relating learning to 
students' previous experiences, knowledge or interests, 
Problem Based Learning, field trips, use schoolyard for 
lessons, encouraging reflection, hurricanes, global 
warming) 


2 




Collaborative Learning (arrange students in flexible 
groups w/ assigned roles to work on various tasks, e.g. 
conducting lab/field activities, inquiry projects, group 
science fair projects, discussion, heterogeneous.) 


2 




Questioning (varying time, positioning, or cognitive levels 
of questions, e.g. increasing wait time, adding pauses at 
key student-response points, including more high- 
cognitive-level questions, stopping visual media at key 
points and asking questions) 


1 


Many choral response 
with little wait time 


Inquiry (student-centered, inductive instructional 
activities, e.g. using guided or facilitated inquiry activities, 
guided discoveries, inductive lab activities, indirect 
instruction. Using Descriptive, Comparative or 
Experimental designs.) 


1 

1 




Manipulation (opportunities to work or practice with 
physical objects, e.g. operating apparatus, developing 
skills using manipulatives, drawing or constructing 


1 
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something) 






Testing (changes in frequency, purpose, or cognitive 
levels or evaluation, e.g. providing immediate or 
explanatory feedback, using diagnostic testing, formative 
testing, retesting, testing to master) 


NA 




Instructional Technology (use tech, to enhance 
instruction, e.g. using computers, etc. for simulations, 
modeling abstract concepts, and collecting data, showing 
videos to emphasize a concept, using pictures, 
photographs, or diagrams, wikis, pod casts, blogs) 


NA 




Enhanced Material (modified instructional materials, e.g. 
rewriting or annotating text materials, tape recording 
directions, simplifying lab apparatus to meet student 
needs) 


NA 




Direct Instruction (teach skills, how to use equipment, 
techniques, etc.) 


NA 





4. Interview with Teacher 



A. Typical science lesson? 

B. Was the same lesson taught 
previously? 

1. What are some examples of either 
CSM or integrated lessons you used 
in your classroom? 

2. How well did it/they work? How did 
you measure its effectiveness? Will 
you teach it again? How would you 
change it to teach again? 

3. How many CSM or integrated 
lessons have you &/or your team 
done this year? 

4. If none, then why? 

5. How else can we help you either 
correlate or integrate science & 
math? 

6. What did your Principal do to help? 

7. What, if any, changes this year in 
your classroom or test scores 
(individual classroom tests or 
district benchmarks) or any area can 



Typical concept, but not typical lab. 1 st used this 
year, but saw it used & thought it was great. 



Use again, but not use M&Ms. Students not 
understand cause confuse 



Language, similarities/differences, compared 
formulae charts, review sci. TAKS test & see 
how math related questions Holt Sci. text. Just 
copy & put onto cards. Sci T not have time to 
use in sci., but math T can because math dbl 
blocked. Similarities & Differences. Plan during 
lunch & team times. Team cross disciplines 
more now w/ ESL & Reading. Hard to team- 
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you discern? 
C. 


teach w/ facilities. Perhaps use cafeteria. New 
addition will have accordian doors so perhaps 
be able to do next yr. 

Conduction/Convection/Radiation sci. lesson w/ 
math T graphing sci. data. Ss seem more 
engaged because they are alerted that both sci. 
& math T talk. Meeting w/ other schools during 
summer & AY help to discuss how to solve 
problems in different ways. C-Scope not teach 
lconcept/lesson & not enough practice. Have to 
follow S&S, but not all lessons. Nov. 
benchmark. 

Materials -whiteboards, books, NCTM 
standards, demo lesson w/ NCTM standards. 
Principal not as involved as other school 
principals. Ts spending time, but Principals not 
making equivalent effort by coming to AY 
sessions, ask ?s about the training, more 
observations, clarification from ISD about 
objectives on the chalkboard & the conflict w/ 
inquiry strategy. Want to do a team teach. 

Math T need brainstorm of how to use math 
manipulatives in science. Hear faculty input on 
topics like benchmarks, TAKS tutorials put math 
& science together or separate, 

C-Scope S&S. 


D. Which of those would you attribute to 
the Mix It Up training? 




E. What changes to the science and/or 
math program have occurred this 
year? 




F. Which of those would you attribute to 
the Mix It Up training? 




8. What were your overall goals with 
this teacher this year regarding 
math or science instruction? 




G. What goals did you accomplish this 
year concerning math or science 
instruction? 
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H. Of those goals, describe which are 
ones that you feel were most 
influential in student success in math 
or science learning? 




1. What evidence of student learning in 
science or math do you have for this 
cohort of students (eg how well your 
students did in 7 th grade last year 
compared to 8 th grade this year)? 




J. How has student academic 

performance increased for this class 
(eg. Last year 8 th grade compared to 
this year 8 th grade)? 




K. To what extent do the disaggregated 
data show a reduction in achievement 
gaps in math or science? 




L. What was your biggest challenge in 
instruction for math and/or science 
this year? 




M. What was your greatest tool in 
overcoming that challenge? 




M. What percent of your school budget is 
for science laboratory equipment for 
last year and for this year? 




N. What percent of your school budget is 
for science laboratory supplies for 
last year and for this year? 




0. What percent of your school budget is 
for math equipment for last year and 
for this year? 




P. What percent of your school budget is 
for math supplies for last year and 
for this year? 




What was the most valuable part of the 
program for you? 




How can the program be improved? 










167 



Sample 2, Spring 2010: 



TQ “Mix It Up”; Correlated Science and Math 
OBSERVATION FORM 

Teacher G Grade Level 5 Class Size 10 Room Size 

District A School A Principal Z 

Date of Observation 5-13-10 Lesson Topic TAKS Remediation TQ Topic 



Debrief w/ T, Principal, PI & coPI. (P is being removed & moved to CO as head of PR 
because Ts complained to Superintendent about P) 

Reviewed prime & composite numbers for Ss who failed 5 th grade math TAKS. Ss had 
to list 1 st 5 prime numbers after 1. Ss went to board w/ races on 2x2 digit multiplication 

and long division. T had chant - “When I say , you say “ However, T 

had Ss say “When I say prime, you say no factors.” During debriefing, T said that she 
knew that as stated (no factors) is incorrect, but it was part of the correct way & to help 
Ss remember the vocabulary. T thinks that Ss were confused on concept names so this 
was her way to help them remember. PI not convinced that T understood that this 
method was potentially hazardous because Ss may remember “no factors” when they 
should remember “a prime number has 2 distinct factors, 1 & itself.” No manipulative 
nor pictorial representations were used. Choral responses primarily. 

P Buchanan sat in debriefing and commented that Ss are likely to remember the incorrect 
“no factors.” 

T also complained about Ss side-ways multiplying and using “chicken foot” method of 
multiplying incorrectly. Not Best Practice to use a “trick” rather than developing a 
conceptual understanding. However, side-ways multiplying is coming from lower grade 
levels. P said there needs to be some conversation about it & later brought in the 4 th 
grade lead math T to discuss the problem and begin the conversation about why the 
wrong method is being taught at some lower grade level. 

T has great rapport with Ss who did everything she asked them to do even if they didn’t 
want to do it. High energy T & class. 

Recommendations: Plan the review to explicitly address specific objectives with 
multiple representations beginning with concrete activities using manipulatives, then 
pictorial, then abstract (numbers). Use more Best Practice strategies w/remedial students 
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and ELPS if appropriate. Check for understanding calling on non-volunteers and asking 
Ss to explain their work. 



1. Was the concept appropriate to integrate? No, only math skills were being reviewed. 

2. If so, was the lesson integrated? NA 

a. Was each discipline taught conceptually? Math not taught conceptually 
Recommendations: See above 

b. Was the Language of each discipline correct? NA 

c. Was the link between the science and math appropriate/natural? NA 

3. Rubric: Best Practice instructional strategies are measured on a 3-point scale ranging 
from being observed 0 (not at all) to 3 (greatest extent). 



Effective Strategies 


0-3 


Comments 


Enhanced Context (real world, science fair, 
problem/case based, use tech, to bring in real world, 
relating learning to students' previous experiences, 
knowledge or interests, Problem Based Learning, field 
trips, use schoolyard for lessons, encouraging reflection, 
hurricanes, global warming) 


0 




Collaborative Learning (arrange students in flexible 
groups w/ assigned roles to work on various tasks, e.g. 
conducting lab/field activities, inquiry projects, group 
science fair projects, discussion, heterogeneous.) 


1 


Unstructured, ask 
neighbor, 


Questioning (varying time, positioning, or cognitive 
levels of questions, e.g. increasing wait time, adding 
pauses at key student-response points, including more 
high-cognitive-level questions, stopping visual media at 
key points and asking questions) 


2 




Inquiry (student-centered, inductive instructional 
activities, e.g. using guided or facilitated inquiry 
activities, guided discoveries, inductive lab activities, 
indirect instruction. Using Descriptive, Comparative or 


NA 
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Experimental designs.) 






Manipulation (opportunities to work or practice with 
physical objects, e.g. operating apparatus, developing 
skills using manipulatives, drawing or constructing 
something) 


0 


Should have used color 
tiles, rainbow cubes 


Testing (changes in frequency, purpose, or cognitive 
levels or evaluation, e.g. providing immediate or 
explanatory feedback, using diagnostic testing, 
formative testing, retesting, testing to master) 


3 




Instructional Technology (use tech, to enhance 
instruction, e.g. using computers, etc. for simulations, 
modeling abstract concepts, and collecting data, 
showing videos to emphasize a concept, using pictures, 
photographs, or diagrams, wikis, pod casts, blogs) 


NA 




Enhanced Material (modified instructional materials, 
e.g. rewriting or annotating text materials, tape 
recording directions, simplifying lab apparatus to meet 
student needs) 


3 




Direct Instruction (teach skills, how to use equipment, 
techniques, etc.) 


2 





4. Interview with Teacher 



N. Typical math lesson? TAKS review for students who failed TAKS. 

O. Was the same lesson taught previously? 

5. What are some examples of either Integration not appropriate for this lesson. 
CSM or integrated lessons you used in 

your classroom? 

6. How well did it/they work? How did 
you measure its effectiveness? Will 
you teach it again? How would you 
change it to teach again? 

7. How many CSM or integrated lessons 
have you &/or your team done this 
year? 

8. If none, then why? 

9. How else can we help you either 
correlate or integrate science & math? 

10. What did your Principal do to help? 
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11. What, if any, changes this year in your 
classroom or test scores (individual 
classroom tests or district 
benchmarks) or any area can you 
discern? 




P. What was your biggest challenge in 
instruction for math and/or science this 
year? 


Moving from self-contained classroom where 
T had control over the time schedule to 
departmentalized where the timeframe was 
structured and T has no control. T very 
frustrated. 


Q. What was your greatest tool in 
overcoming that challenge? 


Time to get accustomed to it. 


What was the most valuable part of the 
program for you? 


Become more comfortable w/ teaching only 
math 


How can the program be improved? 


T focused on time issue which program can 
help. 
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Appendix G 
Principal Reflections 



NAM E : Principal T 

Journal your impressions: 

Day 1 

How can you use teacher and student science and math performance data to improve 
instruction? 

After analysis and presentation, plan areas of focus with staff, work the plan , assess 
along the way, evaluate. 

How will you facilitate science and math integration in your school? 

Through PLC’s 

What student engagement strategies can also be useful in other content areas? 

Small groups - use of manipulatives - think pair share 

Day 2 

What changes in your teacher observations can you make to improve science and math 
integration? 

Focus on one or two areas 

What is your action plan to help your math and science teachers collaborate? 

Model and be there for the teachers 

How has your view of inquiry been expanded? 

Tremendously 

How will you know if the Mix-It Up program has been successful for you and your 
teachers? 

I intend to do a control group as well. 
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NAM E : Principal U 

Journal your impressions: 

Day 1 

How can you use teacher and student science and math perfonnance data to improve 
instruction? 



How will you facilitate science and math integration in your school? 



What student engagement strategies can also be useful in other content areas? 



Day 2 

What changes in your teacher observations can you make to improve science and math 
integration? 

Make it a priority, it will drive our PBL instruction 

What is your action plan to help your math and science teachers collaborate? 

The DuFour model of continuous improvement (modified) 

How has your view of inquiry been expanded? 

I see how absolutely critical it is to good science math instruction. We will incorporate 
inquiry into all subject areas. 

How will you know if the Mix-It Up program has been successful for you and your 
teachers? 

I will observe then and their interactions with students and review their LPs and analyze 
their data. 
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NAM E : Principal W 

Journal your impressions: 

Day 1 

How can you use teacher and student science and math perfonnance data to improve 
instruction? 

Come up with action plan for collaboration. 



How will you facilitate science and math integration in your school? 

Plan meeting, education action research with data support. 

Teacher planning, collaboration with teacher in the core subject. 

What student engagement strategies can also be useful in other content areas? 

Hands on across all subject matter. 5E plan for all learning. 

Hands on, presentation based on a concept, shared performance, use of a rubric. 

Day 2 

What changes in your teacher observations can you make to improve science and math 
integration? 

Be specific in observations; take lesson plans to walk 

What is your action plan to help your math and science teachers collaborate? 

We have already allowed them to meet but not at this time to look at data per the 
discussion 

How has your view of inquiry been expanded? 

All is new information for me 

How will you know if the Mix-It Up program has been successful for you and your 
teachers? 
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NAM E : Principal X 

Journal your impressions: 

Day 1 

How can you use teacher and student science and math perfonnance data to improve 
instruction? 

By working together with teachers as we learn together through constant collaborative 
type activities, focus on metacognitive reflective practices and allow teachers 
unconditional space to fail. 



How will you facilitate science and math integration in your school? 

I will guarantee that my teachers use the collaboratively planning time to focus on the 
lesson study process so that they can deepen their content knowledge. 



What student engagement strategies can also be useful in other content areas? 

I think the focus on the process and the move from 5E to 7E is a valuable process in all 
content areas. 

Day 2 

What changes in your teacher observations can you make to improve science and math 
integration? 

Use a rubric that priorities the teacher and student behaviors that are to be observe 
focusing on hands on activities and modified student directed tasks, activities and 
assignments. 

What is your action plan to help your math and science teachers collaborate? 

We will create within our TAP cluster a PLC environment where teachers meet 
collaboratively 90 min each week and 120 min each month. 

How has your view of inquiry been expanded? 

WOW! Exponentially! WOW! 

How will you know if the Mix-It Up program has been successful for you and your 
teachers? 
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NAM E : Principal Z 

Journal your impressions: 

Day 1 

How can you use teacher and student science and math perfonnance data to improve 
instruction? 

Data should be used to target instruction and to prepare needs assessment. 



How will you facilitate science and math integration in your school? 

Being supportive of my teachers and giving them every opportunity to leam about and 
implement science and math integration. 



What student engagement strategies can also be useful in other content areas? 

Making things relevant 

Day 2 

What changes in your teacher observations can you make to improve science and math 
integration? 

Use domains to focus on what teacher is doing to implement the 5E model 



What is your action plan to help your math and science teachers collaborate? 

To come up with a systematic plan to collaborate with teachers 
How has your view of inquiry been expanded? 

I understand that inquiry should incorporate the 5E model 

How will you know if the Mix-It Up program has been successful for you and your 
teachers? 

Evidence of 5E model in the classroom during walk-throughs and PLC’s. 
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Appendix H 
Principal Interviews 

4-1-10 Interview Principal X 

Mix has enabled more collaboration, paying attention to science & math language 

Ts have 90 min. plan til spring break. Now have only lunch. 

But, focused on management problems. 

Teacher Q having many problems. 

Teacher I modeled one science lesson for Teacher Q. 

More dialog between Teacher I & Teacher Q after each Mix training. 

Teacher Q - has science improvement plan where Ss are struggling. - strategies to 
address. 

Pre-test for Tier 1-90 min core, Tier 2 & 3 additional time (30 min) for Sci & SS til 
spring break. Interventionist comes in & works w/ Tier 2 & 3 Ss. Now working on Math 
& Science. 

Teacher I is managing & teaching 6, 7, 8 math. Teacher I is managing sci. 

Mix Impact 

Vocabulary - Ts paying more attention to their use of vocabulary in science & math and 
not being sloppy w/ their instructional language so Ss are not mis-led. 

Teacher I brings back many ideas from Mix & discusses w/ Teacher Q. 

Teacher Q - increased content, but not class management & using mostly verbal, not 
visual & multiple ways to solve math problems. Standards not clear in lessons, good 
feedback to Ss, but Ss not fully attentive. Excused 2 Ss because they did well on 
Benchmarks - not lend insight where S explains how solved problem. All auditory. 

Not Video tape. 

Every Tu. Cluster meeting to look at student work. 

2 alt cert teachers not last til end of probationary period due to poor management. 

How can Mix help? 

Management 

How to use PPt to plan lesson - get in questions, sequence, effective time management, 
equity, 

Videotape lesson in summer 

Mix did help Principal ?- by teaching Ps how to use math lesson tapes, (Good Morning 
Ms. XYZ), how to better support Ts, how to debrief w/ teachers, 

Our suggestions: 

7E lesson plan format used by HCK may be inhibiting the lesson plan. 

Need to videotape lessons & critique 
May - 

Bring S&S, materials for 1 st 6 weeks. Summer - bring Math & Science S,& S 
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suggest Ts team plan, video lessons, write reflection, view video, write video 
reflection, meet & T gives reflection. 

Teacher Q may not be retained. (She was not retained) 



5-14-10 Interview Principal Y 

Flex Schedule - Ss who pass TAKS get out of school May 19 
170 days - no staff waiver days 

Flex schedule problems 
TIER Ss get TAKS remediation 

Ss who want to stay can. Ss can choose an activity or TAKS prep 
Teacher P’s science unit - Space, final frontier, 90 min. blocks 
Inquiry Circles - teacher, teacher, teacher 

Problems w/ TAKS Obj. 2,4,6 - need activities for Ts 
Obj 2 = algebraic reasoning 
Obj 4 = measurement 

Obj 6 = processes & tools (problem solving) 

Mix ’10 training will focus on algebraic reasoning & incorporate measurement and 
problem solving. 

School D uses “Curriculum on Wall” to encourage interdisciplinary conversations - 
where connect to high interest courses - Ex. how can shop T get better understanding of 
math to make connections? Mix teachers (Teacher P & Teacher O) report that using 
Curriculum on Wall helped them to move better into integrating science & math 

Next yr Ts have to match to CScope more. However, CScope Science will not be 
available next year. Only the S & S and unit outlines will be available. 

Principals have alerted the Ts to download the CScope lessons to use next year. 



5-14-10 Interview Principal Z 

P being moved to ISD PR Dept, in ISD due to a few T complaints & there is no ID of 
new P. 

P talked to 4 th grade lead math T re some problems that 5 th grade math T (Teacher G) was 
seeing & reported during the observation debriefing. 4 th grade lead math T met w/ [CSM 
researcher] to discuss the problems to begin the discussion about where the incorrect 
math procedures (sideways & chicken foot multiplying) are emanating. 4 th grade T says 
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2 digit #s not taught til after TAKS & as far as she knows the wrong methods are not 
taught, but will have conversation w/ other 3 rd & 4 th grade Ts. 

Vertical term meeting only 1/yr. 




